ai newsMo., 19. Okt.
EilmeldungOne Repo, Two Rulebooks: Nested AGENTS.md for Codex, Subfolder CLAUDE.md for Claude Code

We Ran 8 Texts Through an AI Detector. It Called a Human Wikipedia Paragraph 99% AI.

AI detectors are everywhere now — in classrooms, in hiring, and in freelance contracts that let clients reject work when a tool flags it. The pitch is simple: the detector can tell whether a human or a model wrote a text. So we ran a small, fully disclosed test: four human-written texts and four mo

dev.toMontag 16:25 · 19.10.2 Min. Lesezeit
Bild: dev.to

AI detectors are everywhere now — in classrooms, in hiring, and in freelance contracts that let clients reject work when a tool flags it. The pitch is simple: the detector can tell whether a human or a model wrote a text.

So we ran a small, fully disclosed test: four human-written texts and four model-written texts, all between roughly 110 and 160 words, through one detector.

One of the human texts came back 99% AI.

That text was the opening of Wikipedia's article on photosynthesis.

The detector: the Clever AI Detector — the same widget that powers the free detector on our own site. We didn't run this to promote it or to attack a competitor; we ran it on the tool our readers actually use, and we're reporting what it did, including the false positive.

The AI texts (4): a social-media marketing blurb, a product description, an internal policy email, and an opinion-essay intro. All generated by MiMo V2.6 — the model used in RewriteAI's editorial workflow — at default sampling settings, in a single unedited pass, specifically for this test. Disclosing that is the same policy we tell writers to follow.

The human texts (4): chosen to span different registers of real writing, all license-safe to quote:

Nothing was edited, translated, or smoothed. Every text went in exactly as shown, within the widget's stated 80–3,000-word range. All scores below were recorded on October 17, 2026.

Two clean findings and one uncomfortable one:

This is the part I find most interesting from a modeling standpoint: detectors don't inspect provenance. They model the statistics of the text — roughly, two properties:

Encyclopedic prose is engineered to be low-perplexity and low-burstiness: neutral register, standard constructions, terminology used in its most statistically expected sense, uniform sentence length, content words distributed with unusual evenness.

Those are precisely the properties a language model produces by default — and precisely what detectors learn to flag.

In other words, the detector likely wasn't detecting "AI authorship" so much as detecting "textbook register." Wikipedia shares its statistics with machine text. Austen and Thoreau don't. The arXiv abstract — human, dense, but written with a distinctive authorial hand — landed at 12%.

This is the same failure mode Liang et al. documented at scale when detectors systematically mis-flagged non-native English writers: careful, constrained prose reads as machine prose, and the writers least able to defend themselves absorb the penalty.

Eight texts is a tiny sample — this is an illustration, not a benchmark, and it cannot produce an accuracy rate. But it can show, concretely, the failure mode the research literature warns about.

Full write-up, methodology, and all eight scores: our full false-positive test on RewriteAI

Working with detector output in your own pipeline? RewriteAI also has RewriteAI's AI humanizer, built around the same premise: review the output, don't just trust a score.

Quelle: dev.to (dev.to). Original öffnen