Llm judge model

LLM judge model. I test whether the AI actually helps students learn, not whether it just sounds right. Education judgment comes first for me. Then I build runnable AI-quality harnesses that make that judgment measurable. This page walks through the method, and links to one you can point at your own system.

This is the method behind Can you trust the model that grades your content? Measuring when an AI judge waves through broken work. You can read it and run it yourself.

1. Make the failure observable.

I pick the specific way something can go wrong, then build the smallest input that triggers it. If I can’t make it fail on purpose, I can’t prove it works either. I ran this check in Measuring feedback integrity: a blind-solver that catches AI explanations leaking the answer.

2. Measure against a baseline, not a vibe.

I compare the result against a neutral control, so the number actually means something. A score with nothing to compare it to doesn’t tell you anything. I ran this check in Same answer, different grade: measuring when an AI grader can’t hold a verdict.

3. Check it a second, independent way.

I re-run the test with a different model family, or against a held-out set. Agreement across independent checks is the only verdict I trust. I ran this check in Measuring feedback integrity: a blind-solver that catches AI explanations leaking the answer.

The overlap is what I bring: classroom, curriculum, and assessment judgment made measurable with AI and eval tooling. Every claim on this page maps to a public, reproducible harness, not a slide. If you want this run against your own system, the method transfers directly.