General Audience: AI Illustration Benchmarking

We're running a paid study to benchmark the visual quality and accuracy of AI-generated illustrations. By evaluating these images head-to-head, your feedback directly helps us understand which models produce the cleanest and most accurate results. We want your honest, gut-level preferences to guide our assessment of these generated visuals.

You will complete a remote task where you view a short text prompt alongside two AI-generated illustrations. For about 5 minutes, you will evaluate 20 or more matchups to decide which image better matches the prompt and has higher visual quality. You can choose whether the left or right image is better, declare a tie, or reject both if they are unacceptable. The flow includes built-in calibration checks to ensure attentive reviewing, and you can optionally use keyboard shortcuts to move quickly.

This opportunity is open to a general audience with a keen eye for visual detail. We welcome everyday internet users, hobbyists, and professionals from any industry. No formal art or design experience is required, as we are strictly looking for your genuine, gut-level taste.

• Review a short text prompt and compare two AI-generated illustrations side-by-side.
• Evaluate each matchup based on prompt accuracy and overall visual quality.
• Select a winner, declare a tie, or reject both images for approximately 20 or more comparisons.
• Complete calibration checks accurately by identifying intentionally broken or blank images.

• Willingness to provide honest, gut-level feedback on visual media.
• Strong attention to detail to catch visual glitches or weird artifacts.
• Ability to maintain focus for a continuous 5-minute evaluation session.
• No professional art or design background required.

• You will be engaged as an independent contractor.
• This is a fully remote opportunity that can be completed on your own schedule.
• Opportunities can be extended, shortened, or concluded early depending on needs and performance.
• Your participation will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are processed weekly based on services rendered.
• We are unable to support H1-B or STEM OPT candidates at this time.