Content Evaluators: Building Rubrics for Digital Artifacts

We're running a paid study on how professionals assess the quality of complex digital artifacts. Establishing clear rubrics helps ensure consistent evaluation across varying types of generated content. Your insights will directly inform the structural standards we use for future assessments.

During this remote session, you will review several examples of digital artifacts and propose grading criteria for each. You will walk us through your thought process as you define what makes an artifact high or low quality. We will ask you to draft a sample rubric and apply it to a test case. Finally, we will discuss how to refine these rubrics to handle edge cases effectively.

We welcome experienced content evaluators, instructional designers, and domain experts accustomed to grading complex work. Professionals with a background in quality assurance, AI evaluation, or technical writing are highly encouraged to apply. You should be comfortable building structured rubrics from scratch and explaining your methodology clearly.

• Review sample digital artifacts to identify key quality indicators.
• Draft comprehensive grading rubrics based on provided criteria.
• Apply your custom rubrics to test cases and explain your scoring.
• Discuss strategies for handling edge cases and ambiguous content.

• Professional experience in content evaluation or quality assurance.
• Familiarity with creating structured grading rubrics.
• Strong analytical skills for assessing complex digital content.
• Comfortable articulating your evaluation methodology over a remote call.

• You will be engaged as an independent contractor.
• This is a fully remote opportunity that can be completed on your own schedule.
• Opportunities can be extended, shortened, or concluded early depending on needs and performance.
• Your participation will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are processed weekly based on services rendered.
• We are unable to support H1-B or STEM OPT candidates at this time.