no-fabricate gate: every statistic traces to a source, or the build fails
AI can write your copy. The harder question is whether it also wrote your facts. no-fabricate gate is a small, read-only check that extracts every statistic from your content, percentages, dollar figures, multipliers, ratios, and scale words, and fails the build if any of them has no matching entry in a cited source manifest. A number nobody sourced does not ship. It is the sister gate to the de-AI gate: that one catches machine-written style, this one catches machine-invented facts.
The problem: LLM-drafted copy invents numbers
A model asked to write a persuasive paragraph will produce a persuasive statistic, because that is what persuasive paragraphs look like. It is not malice, it is autocomplete. The gate's own README opens with the canonical example: a draft that claims "studies show 73% of teams" when no such study exists.
Once published, a fabricated figure is a liability you cannot see, because it reads exactly like a real one. No human reviewer reliably catches it at scale, and an AI reviewer has the same failure mode as the AI writer. The fix has to be mechanical, not editorial.
The mechanism: a source-pack manifest gates the build
You maintain a manifest of the cited facts your content is allowed to state: each entry is a claim, the exact value, and the source URL. The gate extracts every data-claim from your files and checks each one against the manifest. Any statistic with no matching value fails the build with a hard exit code.
Matching is case-, comma-, and whitespace-insensitive, so a figure split by inline markup in served HTML still matches its manifest value, and a comma-grouped number matches its plain form. Structural numbers, years, ordinals, listicle counts, and section references, are exempt automatically. Your own prices and version strings go in an ignore list, because they are yours, not claims about the world.
The design is fail-closed: a malformed or missing manifest is an error, not a pass. The manifest requires the claim and the URL alongside each value, so it stays a citation record rather than a bare allowlist of numbers.
Install and run
Clone it and point it at a manifest and your content. It scans .html, .htm, .md, and .txt files, needs Node 18 or newer (or Bun), and has zero dependencies.
- git clone https://github.com/BryanBenner/no-fabricate-gate.git
- node cli.mjs --manifest sources.json ./public/articles ./public/guides
- node cli.mjs --manifest sources.json --warn-only ./public (non-blocking baseline audit)
Proven in production, not just published
This gate is the published form of a rail our own autonomous content engine runs on every article it publishes. The pipeline shape it enforces: a research step produces the source-pack with real figures and real URLs, the writing step is told to use no number that is not in the manifest, and the gate runs pre-publish to block anything the writing step invented anyway. The third step is what makes the first two honest. A prompt rule without a gate is a suggestion.
The engine, the extraction shapes, the normalization, and the fail-closed manifest validation are covered by the repo's test suite, 16 tests you can run yourself. And this page is one more instance of the discipline: every statistic in this copy, including the example figures quoted from the README, traces to an entry in this site's own source-pack manifest, and the page fails our build if that ever stops being true.
Keep reading, then get the source.
- Why AI invents numbers, and how to catch it
- Source packs: the manifest that gates your build
- Why it is safe to install
- Sister gate: de-AI gate, for machine-written style
- View the source on GitHub
no-fabricate gate is open source under the MIT license. Read the code, run the tests, and wire it in front of your publish step.
Get no-fabricate gate on GitHubFAQ
Does no-fabricate gate verify that my sources are true?
No. It verifies traceability, not truth: every statistic in your copy must match a value in a manifest that names the claim and the source URL. Whether the source itself is good is a human judgment. What the gate guarantees is that no number reaches production without someone having put a citation next to it.
What counts as a statistic?
Five shapes: percentages, money figures, multipliers, "N out of M" ratios, and scale words such as million or billion. Structural numbers are exempt automatically: years, ordinals, listicle counts, and section references never need a citation. Your own prices and version strings go in the manifest ignore list.
Does it send my content anywhere?
No. It runs deterministic regex over local files, makes no network calls, stores no credentials, and writes nothing. It is invoked explicitly and exits with a code, so nothing about it runs on its own.
How is this different from the de-AI gate?
de-AI gate catches the style of machine writing: the em dash (U+2014), the curly quote, the phrase tells. no-fabricate gate catches the facts of machine writing: statistics no one sourced. They share the same design, deterministic, read-only, hard exit codes, and together they gate both what AI-assisted copy sounds like and what it claims.
What does it cost, and what does it need?
Nothing. It is open source under the MIT license, needs Node 18 or newer or Bun, and has zero dependencies to install.