Experienced Software Engineers: AI Coding Task Creation and Validation

We're hiring experienced software engineers to help us create and validate advanced coding tasks for AI systems. This ongoing work feeds directly into the development of robust evaluation environments for AI agents. Building these benchmarks ensures that emerging tools can handle complex software engineering challenges.

You will design realistic software engineering problems that an AI must solve within a terminal environment. This involves setting up the development workspace, working with Docker, and building complete reference solutions. You will also write clean automated tests to verify that each task can be graded reliably. All work is conducted remotely using standard command-line tools and GitHub.

We welcome backend developers, full-stack engineers, and systems programmers with strong architectural fundamentals. This opportunity is best suited for professionals who are highly comfortable navigating unfamiliar codebases independently. You must have hands-on experience with containerization, automated testing frameworks, and advanced version control workflows.

• Design self-contained software engineering problems for AI evaluation.
• Configure development environments and containerize them using Docker.
• Write deterministic automated tests to ensure reliable grading.
• Develop complete reference solutions for the challenges you create.

• Experienced software engineer comfortable working in unfamiliar codebases.
• Strong proficiency with GitHub, command-line tools, and Docker.
• Demonstrated ability to write comprehensive automated test suites.
• Willingness to complete a one-time software engineering skills assessment.

• You will be engaged as an independent contractor.
• This is a fully remote opportunity that can be completed on your own schedule.
• Opportunities can be extended, shortened, or concluded early depending on needs and performance.
• Your participation will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are processed weekly based on services rendered.
• We are unable to support H1-B or STEM OPT candidates at this time.