Research Scientist / Research Engineer - Safeguards, Controls, and Mitigations (AI Safety Institute)

ABOUT THE SAFEGUARDS TEAMAs AI systems get more powerful and integrated into society, the safeguards that prevent a bad actor from misusing a system will become more important. The AI Safety Institutes Safeguard Analysis Team assesses the susceptibility of advanced AI systems to attack, including by testing system safeguards. The Safeguard Analysis Team is different to other AISI research teams it evaluates attacks that unlock unsafe model capabilities, as opposed to evaluating unsafe capabilities themselves. The team takes a holistic view of security, meaning both ML-based and traditional security attacks and protections are in-scope for the Teams work.The Teams priorities include research-oriented responsibilities like assessing the threats to frontier systems and developing novel attacks and engineering-oriented ones, such as building infrastructure for running evaluations. ABOUT THE ROLE The Safeguard Analysis Team is hiring specialists with skillsets that lean in the direction of either or both of Research Engineer and Research Scientist. We're hiring for all of our sub-teams: Emerging Threats research how current and future advances in the functionality of frontier systems changes AISIs assessment of their security against misuse. Evaluations & Red Teaming build and run AISIs suite of evaluations to measure how easily the protective measures of frontier AI systems can be defeated.Were hiring at various seniority levels within the team, including in Senior Research Engineer / Research Scientist positions. Calibration on final title, seniority and pay will take place as part of the recruitment process. As a Research Scientist/Research Engineer in the Safeguard Analysis Team, you will be part of the team that defines AISIs approach to measuring and improving frontier AI security. You will contribute to a research portfolio that makes frontier AI systems more secure, with responsibilities that could include: Designing or executing AISIs safeguard evaluations against frontier AI systems and analysing their outcomes.Contributing to AISIs strategy for measuring and improving frontier AI security, including developing AISIs threat models.Developing novel attacks against AI systems and exploring their mitigations.Understanding the space of AI controls and mitigations and investigating which defences may be best placed to reduce risk.Building tooling to allow more rapid and reliable evaluation exercises.Undertaking research projects to make progress on relevant open questions, like predicting future model vulnerabilities.Working closely with the Safeguard Analysis Workstream Lead, Research Engineers and other Research Scientists.Collaborating with external experts, contractors, partner organisations and policy makers to coordinate and build on external research.