AI Safety Expert - Red Team - AI Trainer

About the Role

Day-to-day, you will be focusing on jailbreaks, prompt injections, misuse cases, and bias exploitation, generating high-quality human data to test and improve AI models. You will also be annotating failures, classifying vulnerabilities, and flagging systemic risks, using taxonomies, benchmarks, and playbooks to ensure consistent testing. Your work will have a direct impact on the development of AI systems, and you will be working closely with other teams to ensure that your findings are actionable and effective.

What You Will Do

• Red team conversational AI models and agents to test their safety and security
• Focus on jailbreaks, prompt injections, misuse cases, and bias exploitation to identify potential vulnerabilities
• Generate high-quality human data to test and improve AI models
• Annotate failures, classify vulnerabilities, and flag systemic risks
• Use taxonomies, benchmarks, and playbooks to ensure consistent testing
• Document your findings and produce reports, datasets, and attack cases for customer action
• Work independently and asynchronously, using your skills and expertise to identify and exploit potential weaknesses in AI systems
• Collaborate with other teams to ensure that your findings are actionable and effective
• Apply structure using taxonomies, benchmarks, and playbooks for consistent testing
• Ensure flexibility and adaptability across projects, using your skills and expertise to adapt to new and emerging threats

What We Are Looking For

• Fluency in English and Bengali
• Prior experience in red teaming, AI adversarial work, cybersecurity, or socio-technical probing
• Strong communication skills, with the ability to explain risks to technical and non-technical stakeholders
• Experience working with AI models and agents, and a strong understanding of their potential vulnerabilities
• Ability to work independently and asynchronously, using your skills and expertise to identify and exploit potential weaknesses in AI systems
• Strong analytical and problem-solving skills, with the ability to think creatively and outside the box
• Ability to document your findings and produce reports, datasets, and attack cases for customer action

Nice to Have

• Experience with adversarial ML, cybersecurity, and socio-technical risk
• Skills in creative probing, such as psychology, acting, or writing, for unconventional adversarial thinking
• Experience working with taxonomies, benchmarks, and playbooks for consistent testing
• Ability to apply structure using taxonomies, benchmarks, and playbooks for consistent testing

Benefits and Perks

• Competitive hourly rate of $20-$22 per hour
• Opportunity to work with a leading company in the AI research lab space
• Remote work arrangement, with the flexibility to work from anywhere
• Access to a community of elite creative and technical talent
• Opportunities for professional growth and development, with a focus on AI safety and security
• Flexible and adaptable work environment, with the ability to work independently and asynchronously
• Access to a range of tools and resources, including taxonomies, benchmarks, and playbooks for consistent testing
• Opportunities for collaboration and knowledge-sharing with other teams and experts in the field

• Make sure you have a strong foundation in AI safety and security, with experience in red teaming, AI adversarial work, or cybersecurity.
• Develop your skills in creative probing, such as psychology, acting, or writing, to enhance your unconventional adversarial thinking.
• Showcase your fluency in English and Bengali, with a strong ability to communicate risks to technical and non-technical stakeholders.
• Highlight your experience working with AI models and agents, and your ability to identify and exploit potential weaknesses in AI systems.
• Be prepared to discuss your experience working independently and asynchronously, and your ability to adapt to new and emerging threats.
• Research mercor and its clients to understand the company's mission and values, and be prepared to discuss how your skills and experience align with these.
• Consider creating a portfolio or github repository to showcase your work and demonstrate your skills to potential employers.