fact-gate
Open-source developer tool — verifies AI-written PR descriptions and READMEs against the diff
An open-source CLI, GitHub Action and Claude Code hook that checks an AI-written PR description, README or changelog against the actual diff, test output or a facts file — and blocks when a number, date or claim doesn't hold up.
93
Tests
100/100
Own eval suite
0
Dependencies
Challenge
AI coding agents round numbers, invent dates and overstate test counts in the text they write about their own changes, and every existing claim-verifier on the market runs a second model to check the first one — adding cost and its own failure mode instead of removing one.
Solution
A deterministic, zero-dependency checker: it extracts checkable claims from text, compares each one against the diff, a measured command/file/JSON value, or the test suite's own output, and exits non-zero on a contradiction — no second model, no API key. Published on npm and the VS Code Marketplace, with 93 tests and a 100/100 pass rate on its own evaluation suite.
Stack
- Node.js
- CLI
- GitHub Actions