Our AI Reviewer Got the Verdict Right and the Explanation Wrong

We are building a local agent that can write short descriptions and topics from source material, then check its own work. The goal is not just to get an answer out. We want it to finish the task without inventing facts, manufacturing problems or damaging something it already got right.

That last part has been harder than it sounds.

During our own tests, the reviewer sometimes reached the right decision and supplied a false reason for it. Other times it rejected a useful answer while missing a defect we could check directly in the text.

The hallucination was not always in the answer. Sometimes it was in the review.

In one of our evaluation cases, a description claimed an improvement that its source did not establish. The source described an activity without measuring the claimed benefit.

The reviewer correctly identified the unsupported result. But it also accused the description of directly copying the source.

That was not what happened. The claimed improvement was absent from the source, and the text was not a copied source sentence. There was a real factual problem, but the copying accusation did not explain it.

We also found a review that reached the correct unsupported-result conclusion while mis-describing the evidence it cited. The cited passage concerned the lack of measurement; the explanation described a different passage about volunteers distributing forms.

Those are actual findings from our tests, not hypothetical examples of how a critic might fail.

This matters if the critic will guide another model's revision. A false copying accusation could encourage the writer to paraphrase the unsupported result instead of removing it.

Counting the rejection alone would hide the difference between finding a problem and accurately explaining it.

The reverse problem appeared too. The reviewer objected to a source-faithful description for omitting a volunteer count and a detail about handing out forms. Neither detail was necessary to support what that description actually said.

We wanted concise summaries, not miniature copies of every source detail. Our evaluation allowed harmless omissions. Treating every missing detail as a factual failure pushed against the task we wanted the agent to complete.

Meanwhile, the same paired test included an actual complete source sentence copied into a candidate. Both reviewer conditions missed that copying defect. They also missed structural violations that a separate program could identify.

That combination was the important finding: false accusations about copying in some outputs, missed copying in another. Turning up the critic's suspicion was not a reliable substitute for getting the distinction right.

We tested 12 inputs under two prompt conditions: the existing instructions and a version with additional procedural guidance. This comparison changed the guidance, not the model weights.

Each condition got six final decisions right. Once we required the checks and explanations to be supported as well, each condition had only three fully qualifying reviews out of 12.

The total stayed the same, but the cases did not. The added guidance gained one fully qualifying review and lost another that had qualified under the original instructions.

Looking only at the improved example would have given us a progress story. Looking at the paired results gave us a different conclusion: this change had not earned promotion.

This comparison was part of several days of development, hands-on human review and cross-family critique. Two additional model families independently regraded all 12 saved candidates and 24 critiques with condition labels and earlier scores hidden. Both reproduced the same fully qualifying results, including the same gain and regression. We preserved disagreements about individual judgments and checked literal and structural claims directly against the source text.

The useful change in our work has been separating questions that a single pass rate can blur together. Did the critic make the right decision? Were its accusations true? Did its cited evidence say what it claimed? What did it miss? Did the change break a case that previously passed?

We kept the failed outputs and the regressions. We did not replace the posting model or call the broader agent qualified because one part of a review looked convincing.

Adding a critic has not removed the need to verify. It has given us another component whose mistakes need to be understood before we let it steer revisions.

The question we are carrying forward is simple:

If the writer followed this explanation, would it fix the actual defect?

A right verdict is a start. A truthful explanation is a separate achievement.

*AI disclosure: This article was drafted by an AI assistant from our own development records.