Deterministic vs perplexity detectors: why false positives happen
Two tools can both claim to catch AI writing and work in completely different ways. A perplexity detector reads a passage and estimates how likely it is to be machine-written. A deterministic gate reads the same passage and reports whether a specific tell is literally present. That difference is why one produces false positives and the other does not, and it is the whole reason de-AI gate is built the way it is.
The false-positive paradox
A perplexity detector scores structure. It asks how predictable the wording is, and treats smooth, predictable text as probably machine-written. The problem is that plenty of careful human writing is smooth and predictable too, so it gets flagged. The failure is not a bug to be patched; it is what scoring probability does.
The evidence is on the record. A 2023 Stanford study by Liang and colleagues found that widely used GPT detectors misclassified about 61 percent of essays by non-native English writers as AI-generated, while rating native-English essays as human. That same year OpenAI withdrew its own AI Text Classifier, citing low accuracy. When the decision is a probability, a share of real human writing is always caught in the net.
Deterministic tell-matching
de-AI gate does not estimate anything. It reads your files and checks them against a fixed catalog of literal tells: specific characters and specific phrases. If the em dash (U+2014) is in the file, that is a fact. If it is not, there is nothing to flag. There is no probability in the loop, so there is no probability to be wrong about.
That is the trade the gate makes on purpose. It will not guess that a passage "feels" AI-written, because guessing is exactly what generates false positives. It reports only what is verifiably present, which is what makes it safe to block a build on.
Side by side
The two approaches differ on every axis that matters for a team deciding whether to trust the result.
| Perplexity / probabilistic detector | de-AI gate (deterministic) | |
|---|---|---|
| How it decides | A probability that the text reads as AI | Whether a literal tell is present in the file |
| False positives on human writing | A known, documented failure mode | None; a match is a real character or phrase |
| Where it runs | A web form, after the writing is done | A pre-commit hook or CI step, before shipping |
| What it needs from you | You paste or upload your content | Local files only; no network, no upload |
| What a flag tells you | Maybe AI, maybe a careful human | Exactly which tell, and where to fix it |
The proof you are reading
This page is itself the demonstration. It argues about AI tells directly. It names the em dash (U+2014), the curly quote (U+201C), invisible unicode, and the set of high-precision phrase tells the catalog flags. A probabilistic detector would likely score a page this dense with tell-talk as suspicious.
And yet this page passes the de-AI gate in this site's own build, because it contains none of the literal tells, only their names and codepoints. A deterministic gate reads what is actually on the page. That is the claim, shown rather than asserted.
Keep reading, then get the source.
- de-AI gate: what it is
- Run it as a CI or pre-commit gate
- Why it is safe to install
- View the source on GitHub
de-AI gate is open source under the MIT license. Read the catalog, run the tests, and see the deterministic approach for yourself.
Get de-AI gate on GitHubFAQ
Why do AI detectors flag human writing?
Because they score probability, not fact. A perplexity detector rates how predictable the wording is and treats smooth text as likely machine-written, so careful human writing gets caught. A 2023 Stanford study found detectors misclassified about 61 percent of essays by non-native English writers as AI-generated.
Is a deterministic gate less capable than an AI detector?
It answers a narrower, checkable question. It does not judge whether a whole document was AI-written; it reports whether specific, literal tells are present. That is a smaller claim, and it is the claim you can safely block a build on, because it is never a guess.
Can I trust it to not block my build over nothing?
Yes, because a hard failure always points at a literal tell in the file, a specific character or phrase you can see and remove. There is no probability threshold to tune and no false positive to argue with.