DevOps Engineer - AI Model Evaluator
WFA Digital Insight
As the demand for skilled DevOps engineers continues to soar, with a 25% increase in remote job postings in 2025, professionals with expertise in cloud infrastructure and AI model evaluation are in high demand. Mercor, a company that connects top talent with leading AI research labs, is looking for a DevOps Engineer to join their team. With the rise of AI-powered technologies, companies are looking for individuals who can bridge the gap between development and operations, making this role a compelling opportunity for those looking to work at the forefront of innovation. Candidates should be prepared to showcase their experience with cloud platforms, Kubernetes, and CI/CD systems, as well as their ability to work with AI coding agents.
Job Description
About the Role
The DevOps Engineer role at mercor is a unique opportunity to work at the intersection of cloud infrastructure and AI model evaluation. As a member of the team, you will be responsible for using frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. This role matters because it enables the company to leverage the power of AI to improve the efficiency and reliability of their cloud infrastructure.The day-to-day responsibilities of this role will involve reviewing model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, and infrastructure automation. You will also be responsible for identifying bugs, edge cases, reliability issues, and failure modes in model outputs, and comparing outputs from multiple frontier models to assess strengths and weaknesses.
The ideal candidate will have a strong background in DevOps, SRE, or Cloud Engineering, with experience working with cloud platforms, Kubernetes, and CI/CD systems. They will also have experience using AI coding agents and evaluating model-generated infrastructure and reliability engineering solutions.
What You Will Do
- Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks
- Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, and infrastructure automation
- Identify bugs, edge cases, reliability issues, and failure modes in model outputs
- Compare outputs from multiple frontier models to assess strengths and weaknesses
- Apply professional engineering judgment to realistic infrastructure engineering scenarios
- Collaborate with the team to improve the efficiency and reliability of cloud infrastructure
- Develop and implement automated testing and deployment scripts
- Troubleshoot and resolve issues with cloud infrastructure and AI models
- Participate in code reviews and contribute to the improvement of the codebase
What We Are Looking For
- 2+ years of professional DevOps, SRE, or Cloud Engineering experience
- Experience with AWS, Azure, GCP, or other cloud platforms
- Experience with Kubernetes, Terraform, CI/CD pipelines, or observability tooling
- Ability to evaluate model-generated infrastructure and reliability engineering solutions
- Strong understanding of cloud infrastructure, networking, and security
- Experience with AI coding agents such as Cursor, Claude Code, Codex, Windsurf, or Gemini CLI
- Strong problem-solving skills and attention to detail
- Excellent communication and collaboration skills
Nice to Have
- Experience supporting production-scale systems
- Knowledge of containerization and orchestration using Docker and Kubernetes
- Experience with infrastructure automation tools such as Ansible or SaltStack
- Familiarity with agile development methodologies and version control systems such as Git
Benefits and Perks
- Competitive compensation tied to accepted work
- Opportunity to work with cutting-edge AI technologies and cloud infrastructure
- Collaborative and dynamic work environment
- Flexible working hours and remote work options
- Access to professional development and training opportunities
- Recognition and reward for outstanding performance
- Comprehensive benefits package, including health insurance and retirement plan
- Generous paid time off and holiday policy
How to Stand Out
- To stand out as a candidate, highlight your experience with cloud platforms, Kubernetes, and CI/CD systems, as well as your ability to work with AI coding agents.
- Make sure your resume and online profiles are up-to-date and showcase your technical skills and experience.
- Be prepared to showcase your problem-solving skills and attention to detail during the interview process.
- Research the company and their use of AI technologies to demonstrate your interest and knowledge.
- Prepare to discuss your experience with infrastructure automation, containerization, and orchestration.
- Be ready to provide examples of your experience with agile development methodologies and version control systems.
- Don't be afraid to ask questions during the interview process to demonstrate your curiosity and interest in the role.
This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.