LLM-as-a-judge means using one model to check another model's output — usually because manual review doesn't scale. It sounds like an obvious fix. In practice, a naive version of this check fails in three specific, documented ways.
1. Correlated blind spots
A judge from the same model family, reading the same framing, tends to fail on the same inputs the producer failed on. The check passes most often exactly when it's needed least.
AIReg-Bench, a benchmark that tests how well LLMs assess compliance with the EU AI Act, measured this directly. Legal experts scored 120 documentation excerpts for compliance; ten frontier models scored the same excerpts. o3-mini's scores exceeded the human expert's score in 54.2% of excerpts, and fell below it in only 1.7%. The judge wasn't neutral. It leaned toward agreement, in the same direction its own training likely leans.
Agreement is the judge's failure mode here, not its success mode.
2. Sycophancy toward the shown answer
A judge that sees the producer's determination before making its own tends to anchor on it. This shows up consistently across published bias audits of LLM judges: a judge given only the input and asked to evaluate blind scores more accurately, on average, than a judge shown the producer's answer and asked to review it. Seeing the answer first changes the judge's answer.
The fix is structural, not a prompting trick: don't show the judge what it's supposed to be checking.
3. The unmeasured validator
This one is the most common and the easiest to miss. If you add an LLM judge to catch errors, but never check whether the judge itself is accurate, you haven't added protection. You've added a second unaudited model and called it a safeguard.
This is the same failure as an unaudited primary system, recurring one level up: the checker becomes as unreliable as the thing it's checking, and nobody notices because nobody is scoring the checker.
The fix: every validator needs its own accuracy score, measured against a small set of examples with known correct answers (a "golden set"). Without that score, a validator is decoration — it looks like a safeguard, but there's no evidence it functions as one.
About Heavy Chain Engineering
Jason Vertrees is the founder of Heavy Chain Engineering (heavychain.org), which helps lower middle-market vertical SaaS companies and PE firms turn scattered AI usage into measurable delivery leverage — 85% faster feature velocity, six-to-eight-week projects shipped in days. If you want help building an AI-native engineering organization, book an AI Delivery Assessment (app.onecal.io/b/jason-vertrees/30-mins).