Finance Professionals: Designing Challenging AI Prompts

We're running a paid study to build a bench of people who are exceptionally good at designing tasks that expose AI model limitations. By turning real-world workflows into demanding requests, we can better evaluate where current models break down. This initial trial helps us identify individuals suited for ongoing prompt engineering and evaluation work.

You will spend about an hour translating a complex workflow from your job or personal life into a demanding prompt that requires reasoning and real-world lookup. After running it in ChatGPT to identify where the model fails, you will refine the prompt until it breaks the system. Finally, you will write a clear grading rubric that a stranger could use to evaluate any AI's attempt at your task. This entire process is screen-recorded, as we are assessing your thought process just as much as the final submitted files.

We welcome professionals, domain experts, and power users who have deep knowledge of specific workflows. You need to be capable of evaluating an AI's output within seconds and comfortable working on a laptop or desktop with a ChatGPT account. Candidates who excel at this trial will be considered for a long-term bench of evaluators.

• Pick a familiar workflow and convert it into a demanding AI prompt
• Test your prompt in ChatGPT to find failure points, making it harder if the AI succeeds
• Write a comprehensive rubric for grading the AI's performance
• Share your screen, camera, and microphone while completing the task
• Submit your prompt, failure notes, rubric, and the generated output file

• Deep familiarity with a specific professional or personal workflow
• Ability to quickly evaluate the accuracy and quality of AI outputs
• Access to a laptop or desktop computer
• An active ChatGPT account
• Comfortable being screen-recorded while thinking through complex tasks

• You will be engaged as an independent contractor.
• This is a fully remote opportunity that can be completed on your own schedule.
• Opportunities can be extended, shortened, or concluded early depending on needs and performance.
• Your participation will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payment is issued as soon as your work is approved.
• We are unable to support H1-B or STEM OPT candidates at this time.