Freelance Agent Evaluation Engineer

MindriftMindrift·Remote(Canada)
Software Development

WFA Digital Insight

As demand for AI and machine learning specialists continues to rise, with a 25% increase in job postings last year, professionals with expertise in software development and AI evaluation are in high demand. Mindrift's innovative approach to building datasets for AI agent evaluation makes this freelance role a unique opportunity for those with a passion for AI and coding. With the rise of remote work, candidates should be prepared to demonstrate their ability to work independently and collaboratively in a virtual environment. Before applying, candidates should ensure they have a strong foundation in Python, full-stack development, and experience with React-based interfaces.

Job Description

About the Role

The role of a Freelance Agent Evaluation Engineer at Mindrift involves creating challenging tasks and evaluation criteria to assess the capabilities of AI coding agents. As a key member of the team, you will work on a part-time, non-permanent project, focusing on developing tasks that simulate real-world scenarios, allowing AI agents to learn and improve their coding abilities. Your expertise in software development, particularly in Python, will be essential in designing and implementing these tasks.

The success of this project depends on the ability to create realistic and challenging tasks that push the limits of AI agents. As such, your work will have a direct impact on the development of AI coding agents, contributing to the advancement of the field. You will be working independently, with the flexibility to manage your schedule, but also collaboratively as part of a remote team.

What You Will Do

  • Design and develop tasks that simulate real-world coding scenarios for AI agents to evaluate and improve their coding abilities
  • Create evaluation criteria to assess the performance of AI agents on these tasks
  • Work primarily with Python, utilizing your full-stack development skills to build robust back-end systems and React-based interfaces
  • Develop and implement automated testing using Docker containers and CI/CD tools to ensure the reliability and efficiency of the tasks and evaluation criteria
  • Collaborate with the development team to integrate tasks and evaluation criteria into the existing system
  • Participate in code reviews to ensure high-quality code and adherence to best practices
  • Troubleshoot issues and optimize task performance
  • Stay up-to-date with the latest developments in AI, machine learning, and software development to continuously improve task design and evaluation criteria
  • Contribute to the development of the project's technical documentation
  • Ensure that all tasks and evaluation criteria are properly documented and easily accessible for future reference

What We Are Looking For

  • Degree in Computer Science, Software Engineering, or a related field
  • At least 5 years of experience in software development, with a focus on Python
  • Background in full-stack development, including experience with React-based interfaces and robust back-end systems
  • Strong understanding of software development principles, patterns, and best practices
  • Experience with automated testing, Docker containers, and CI/CD tools
  • Familiarity with infrastructure tools and version control systems
  • Excellent problem-solving skills, with the ability to analyze complex issues and develop creative solutions
  • Strong communication skills, with the ability to work effectively in a remote team environment

Nice to Have

  • Experience with machine learning and AI development
  • Knowledge of cloud platforms and their integration with software development
  • Familiarity with agile development methodologies
  • Experience with technical writing and documentation

Benefits and Perks

  • Competitive hourly rate of up to $45 per hour
  • Flexible schedule, allowing for remote work and a healthy work-life balance
  • Opportunity to work on a challenging and innovative project
  • Collaboration with a talented team of professionals in the field of AI and software development
  • Professional development opportunities, including access to training and conferences
  • Recognition and reward for outstanding performance and contributions to the project
  • Access to the latest tools and technologies in software development and AI

How to Stand Out

  • Ensure your portfolio showcases your experience with Python and full-stack development, including examples of complex tasks and projects you've worked on.
  • Be prepared to demonstrate your understanding of AI and machine learning concepts, as well as your ability to design and evaluate tasks for AI agents.
  • Highlight your experience with automated testing, Docker containers, and CI/CD tools, as these are crucial for the role.
  • Emphasize your ability to work independently and collaboratively in a remote team environment, with strong communication and problem-solving skills.
  • When negotiating salary, consider the competitive hourly rate and the flexibility of the remote work arrangement, and be prepared to discuss your expectations and requirements.
  • Pay attention to the company culture and values, and be prepared to ask questions about the team and the project during the interview process.
  • Be cautious of any red flags, such as unclear expectations or lack of communication, and be prepared to address any concerns you may have.

This is a remote position listed on WFA Digital, the platform for professionals who work from anywhere. Browse more remote jobs across all categories.