Data Annotation Specialist, Safety

About the Role

The day-to-day responsibilities of this role will involve working independently to label, rank, audit, and refine human- and model-generated text. You will be expected to apply nuanced safety judgment, assessing model outputs against detailed safety guidelines, rubrics, and style standards. Your attention to detail and ability to work independently will be crucial in this role.

Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do, and the company believes that a diverse range of perspectives is a requirement for building great products.

What You Will Do

• Evaluate and improve model safety by labeling, ranking, auditing, and refining human- and model-generated text
• Apply nuanced safety judgment to assess model outputs against detailed safety guidelines, rubrics, and style standards
• Create prompts and safety test cases to evaluate model behavior across safety categories
• Support quality and calibration by identifying annotation inconsistencies or unclear guidelines
• Provide actionable feedback on recurring edge cases, model failures, and opportunities to improve data quality
• Work with precision and independence to complete annotation tasks with strong attention to detail
• Engage with human-generated and model-generated tasks, which will sometimes involve intentional exposure to explicit content
• Use Excel and other tools to manage and analyze data
• Collaborate with the team to improve the overall quality and safety of the AI models
• Stay up-to-date with the latest developments in the field and apply this knowledge to improve the models

What We Are Looking For

• Background and skills in Trust & Safety, Content Moderation, AI model evaluation, or related fields
• Strong contextual judgment, cultural and bias sensitivity, and experience applying nuanced guidelines to complex or ambiguous content
• Ability to work independently with a high degree of independence and make consistent, well-reasoned decisions
• Strong attention to detail and ability to evaluate safety risks across different user intents and scenarios
• Experience with Excel and other data management tools
• Ability to work in a fast-paced environment and adapt to changing priorities
• Strong communication and collaboration skills
• Ability to commit to 16 hours per week minimum at a 45 CAD/hour or 45 USD/hour contract rate

Nice to Have

• Experience with Large Language Models or other AI technologies
• Knowledge of programming languages such as Python or Java
• Experience with data annotation tools and platforms
• Certification in Trust and Safety or a related field

Benefits and Perks

• Competitive contract rate of 45 CAD/hour or 45 USD/hour
• Hazard pay of 15 CAD/hour or 15 USD/hour
• Opportunity to work with a leading company in the AI industry
• Flexible, remote work arrangement with a minimum of 16 hours per week
• Access to the latest technologies and tools in the field
• Opportunity to work on complex and challenging projects
• Collaborative and dynamic work environment
• Professional development opportunities
• Recognition and rewards for outstanding performance

• Develop a strong portfolio that showcases your experience in Trust and Safety, Content Moderation, or related fields.
• Highlight your attention to detail and ability to work independently in your application and during the interview process.
• Be prepared to discuss your experience with Excel and other data management tools.
• Show a willingness to learn and adapt to new technologies and tools, such as Large Language Models.
• Research the company and the role thoroughly to understand the company culture and the expectations of the position.
• Prepare examples of times when you had to apply nuanced safety judgment and make well-reasoned decisions in complex situations.
• Be honest and transparent about your experience and qualifications, and be prepared to provide specific examples during the interview process.