QA Professionals: Evaluate Automated Web Test Failures

We're running a paid study on the reliability of automated website testing tools. We want to understand how accurately our test environments capture real-world web failures compared to expected outcomes. Your feedback directly informs how we calibrate error detection algorithms for future test runs.

You will review evidence from an automated test of a public website through an online interface. We will provide you with the expected outcome, the observed outcome, and two accompanying screenshots for three distinct test cases. For each case, you will analyze the provided data to determine whether it represents a genuine failure and assess its overall severity. You will not need to create any accounts, download software, or visit the live website yourself.

We welcome QA engineers, software testers, front-end web developers, and technical support specialists. The ideal participant is highly comfortable comparing visual screenshots and diagnosing web interface errors. You should have a strong eye for detail and the ability to clearly articulate technical severity.

• Review expected and observed outcomes from automated web tests
• Analyze accompanying screenshots to verify potential visual or functional errors
• Determine whether a flagged discrepancy is a genuine failure or a false positive
• Rate the severity of confirmed test failures based on potential user impact

• Professional experience in software testing, quality assurance, or web development
• Comfortable evaluating visual and functional discrepancies in web applications
• Strong analytical skills to quickly assess the severity of technical failures
• Ability to clearly explain your technical reasoning and diagnostic process

• You will be engaged as an independent contractor.
• This is a fully remote opportunity that can be completed on your own schedule.
• Opportunities can be extended, shortened, or concluded early depending on needs and performance.
• Your participation will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payment is issued as soon as your work is approved.
• We are unable to support H1-B or STEM OPT candidates at this time.