AI Curriculum & QTI Assessment Quality Control: Blind-Solver Checks & Zero Answer Leaks

Shipping a weak blog sentence is annoying. Shipping a test question with two right answers or a tutor hint that gives away Choice B wrecks learner trust in thirty seconds.

I spent fifteen years in classrooms and curriculum leadership before building automated instructional pipelines. Later, I governed 52,000+ multi-agent runs on contract for an AI-education company. Fluent prose hides bad teaching unless you test every question like production software.

Bug 1: Two defensible keys or throwaway wrong choices

Let one model write a multiple-choice item and its own rationale, and it will defend a broken key every time. Blind-solver checks stop that cold. Strip out the key, hand the stem and choices to three separate model vendors, and require all three to pick the exact same answer from scratch.

Good wrong choices aren't random jokes either. Each distractor needs to map to a real mistake learners make on that skill.

Bug 2: Hints that give away the answer

Practice tutors often show a hint after a wrong first try. Because models aim to please, their hints often repeat the exact words sitting inside Choice C. Run a simple leak test to catch it: show a separate model only the hint and the four choices, hiding the question stem. Whenever that blind guesser still nails the right letter, the hint leaked too much.

Bug 3: Silent drift away from the target standard

Ask a model for a tenth-grade question on a specific skill and it will often test side trivia or write at a graduate reading level. Python readability checks, stimulus rules, and binary skill gates block off-target items before human editors waste hours fixing them.

If your team is scaling curriculum or QTI item banks, test the live SCORM & QTI verification demo or book a 20-minute scope call.