← Public failure log
When we get it wrong
A grader bug scored one task per dimension instead of three.
2 July 2026
- What happened
- During pre-launch auditing we found the grader sampling one of three tasks per dimension instead of all three, so composite scores rested on a third of the evidence they should have.
- Root cause
- A loop in the grading path advanced past the remaining tasks in each dimension before they were scored.
- Fix
- The grader was corrected and the rubric version bumped. Every affected score was voided — marked void, never quietly recalculated. Results are version-stamped and history is never rewritten.
- Lesson
- A scoring path gets the same untrusted-input review as payments and auth, and every dimension's task count is asserted in a test.