Image Reviewers: Paid AI Illustration Comparison

We're running a paid study on AI-generated illustrations to benchmark their overall quality and accuracy. This research helps us understand how well different AI models translate text prompts into compelling visuals. By gathering genuine, gut-level preferences from everyday users, we can better evaluate where these models succeed and where they struggle with glitches or poor composition.

You will participate in a remote task where you view a short text prompt alongside two AI-generated illustrations. For about 5 minutes, you will complete 20 or more head-to-head matchups, deciding whether the left or right image is better based on prompt accuracy and visual quality. You can optionally tag reasons for your choice, like cleaner shapes or fewer artifacts, to provide deeper context for your decision. Along the way, you will also complete simple calibration checks to confirm you are actively reviewing the images.

We are looking for everyday users who can provide their honest, gut-level opinions on visual content. No professional art or design experience is required to participate in this task. We welcome students, professionals, and anyone interested in quickly evaluating AI imagery based on basic visual criteria.

• Review a short text prompt and examine two AI-generated illustrations side by side.
• Judge which image better matches the text prompt and possesses higher visual quality.
• Select your preference for each matchup, indicating if one is better, if they are tied, or if both are unacceptable.
• Apply optional reason tags to explain why you chose a specific illustration.
• Complete at least 20 matchups over approximately 5 minutes, remaining attentive to periodic calibration checks.

• Ability to evaluate images based on basic visual quality and accuracy to a written description.
• Comfortable providing honest, gut-level opinions at a steady pace.
• Attention to detail to spot visual glitches, strange artifacts, and intentional blank calibration images.
• No prior art, design, or professional evaluation experience is necessary.

• You will be engaged as an independent contractor.
• This is a fully remote opportunity that can be completed on your own schedule.
• Opportunities can be extended, shortened, or concluded early depending on needs and performance.
• Your participation will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are processed weekly based on services rendered.
• We are unable to support H1-B or STEM OPT candidates at this time.