Rubric Assessment:Grading a Design Rubric
An assessment of whether you can tell a working evaluation rubric from a broken one. You are given a finished presentation deck, the brief that produced it, and a rubric somebody wrote to grade that deck, together with the ten rules they were told to follow. You review the rubric and say what is wrong with it, up to five findings, each ruled on by two other people who read the same packet. The ten rules are published up front, and so are the defects we already found; restating either scores nothing. What is measured is whether the rubric would actually work. No design software needed, about 45 minutes.
The cohort grades itself
Ruling on whether a finding is real needs somebody who read the same material, and the only people who did are the ones who sat the exam. So the people on this board are also the people who checked it.
Who is here
This board is not published yet. Who sat it goes up with the results, not before them.
How the checking works
Every finding needs two independent verdicts, so a cohort that does not check does not get checked. Submitting puts findings in the queue and owes reviews to it, and an attempt that returns less than half of what it owes is left unranked. Precision and rarity are both computed from verdicts other people supplied, and a row ranked on donated labour is not a result.
Checkers never see who wrote what, and are never asked whether a finding is clever, interesting, or better than their own. The only question put to them is whether the thing described actually happens. When two checkers disagree, the finding counts for nobody in either direction.
