Critical Reviewer - Fully Remote | Upto $70/hr

About the Role

The role of a Critical Reviewer is critical in the development of AI models, as it ensures that the output is not only accurate but also nuanced and well-reasoned. Your expertise will help AI models to generate content that is not only informative but also engaging and relevant to the target audience. You will be part of a team of experts who are passionate about AI and its potential to transform industries.

mercors approach to AI model evaluation is unique, as it combines human expertise with machine learning algorithms to generate high-quality content. The company is committed to connecting top talent with leading AI research labs, providing opportunities for professionals to work on cutting-edge projects and contribute to the development of AI models.

What You Will Do

• Evaluate AI-generated responses to ensure accuracy, depth, and nuance in reasoning
• Provide structured written feedback to improve AI model outputs, highlighting areas of strength and weakness
• Identify implicit meanings, gaps in reasoning, and areas for improvement in AI-generated content
• Write clear, precise, and well-evidenced rationales to support your feedback, going beyond surface-level observations
• Work independently to meet deadlines and improve AI model performance, following detailed task guidelines
• Participate in asynchronous work environments, collaborating with other experts to achieve common goals
• Develop and maintain a deep understanding of AI models, their capabilities, and their limitations
• Stay up-to-date with the latest developments in AI research and its applications, applying this knowledge to improve your evaluation skills
• Continuously evaluate and improve your own performance, seeking feedback from peers and supervisors to refine your skills

What We Are Looking For

• Bachelor's degree from a top-500 globally ranked university, preferably in a field related to AI, computer science, or linguistics
• Strong analytical and written communication skills, with the ability to provide nuanced and well-evidenced feedback
• Ability to work independently, following detailed task guidelines and meeting deadlines in a remote work environment
• Strong critical reading skills, with the ability to identify implicit meanings and gaps in reasoning
• Native English fluency, with excellent writing and editing skills
• Experience working with AI models, or a strong desire to learn and develop skills in this area
• Ability to work asynchronously, collaborating with other experts to achieve common goals
• Strong attention to detail, with the ability to provide high-quality feedback and evaluation

Nice to Have

• Experience working in AI research labs, or a related field
• Knowledge of machine learning algorithms and their applications
• Familiarity with natural language processing and its applications
• Experience working in a remote work environment, with the ability to self-motivate and manage time effectively

Benefits and Perks

• Competitive compensation, with an hourly rate of up to $70
• Opportunity to work with leading AI research labs, contributing to the development of cutting-edge AI models
• Flexible work arrangements, with the ability to work remotely and manage your own schedule
• Access to a community of experts in AI research and development, providing opportunities for networking and professional growth
• Ongoing training and development opportunities, helping you to stay up-to-date with the latest developments in AI research and its applications
• Recognition and reward for outstanding performance, with opportunities for career advancement and growth
• Comprehensive benefits package, including health insurance, retirement savings, and paid time off

• Develop a portfolio of your work, showcasing your evaluation and feedback skills, to demonstrate your expertise to potential employers.
• Practice providing nuanced and well-evidenced feedback, using examples from your own work or from publicly available AI-generated content.
• Stay up-to-date with the latest developments in AI research and its applications, attending webinars and conferences to network with other experts and learn about new opportunities.
• Highlight your ability to work independently and manage your time effectively, as these skills are essential for success in a remote work environment.
• Be prepared to provide specific examples of your experience working with AI models, or your desire to learn and develop skills in this area.
• Research the company and its values, demonstrating your knowledge and enthusiasm for the role and the company's mission.
• Prepare to discuss your critical reading skills, providing examples of how you have identified implicit meanings and gaps in reasoning in AI-generated content.