I use ChatGPT or Claude in the browser
- Do my custom instructions do anything? No install, about thirty minutes.
- Is a file someone sent me safe to paste in?
- Did the AI change lately?
No terminal needed for these three.
An AI skill is a file of instructions that people add to an AI assistant, such as Claude, to make it better at a job. Driftproof tests whether a skill really helps, the way a medicine is tested against a sugar pill. The same test works on custom instructions, saved prompts and rules files for Claude Code, Cursor, Copilot and Windsurf. The latest reports give the AI the same tasks three ways: with the skill, with a fake of the same length, and with nothing added. Then it scores every answer, again and again. Every result is a dated receipt with its own page; anyone with a terminal can verify the file.
I use ChatGPT or Claude in the browser
No terminal needed for these three.
I use AI skills
Just curious
I make skills
A piece by Maverick: quoted AI benchmark gaps, checked against their own sampling noise.
Found by Driftproof: the invisible space that AI graders got backwards. Try the quiz.
18 reports published
319 receipts published
12 models measured on
Comparing two benchmark scores? The gap calculator says whether the gap is bigger than the benchmark's own sampling noise.
For the curious
That range is a band. The dated file that records all of it is a receipt. What changes between two receipts is drift.
Move the floor and change what counts as a band to see the verdict change. This is the bundled example receipt's own per-case data.
Walk through one real test, step by step, or try to tell which answer had the skill.
The reports
Placebo comparison report
What moved: a same-length placebo text
claude-fable-5-2
2026-11-11
The latest report reads each skill against a same-length placebo as well as against no skill: a text of the skill's length in bytes, in its place, drawn from a fixed corpus chosen without reference to any task. See your file's fake shows the text Driftproof would use for your file, in your browser, before you measure it. On a run that kept its answers, the results page and driftproof blind show each task's two answers without saying which used your file, ask you to pick, and after your picks say how often you were right beside the judge's scores. In Claude Code, /driftproof:blind opens it.
The same skill on different models and dates: what each receipt read, model by model.
Writing: What happens to agent skills when the model changes, the launch essay reading all eighteen reports on what a moving model does to encoded expertise.
For people who use AI skills
Paste its link and see whether it has been measured, or browse the skills already on record. Look up a skill. Do popular AI skills help? Browse skill badges.
Before you measure a file of your own, open See your file's fake: the same-length text that stands in for the sugar pill.
For people who make skills
A skill's own tests passing is one answer. Driftproof asks two more: whether the scores with the skill and without it separate beyond their spread, and whether that held when the model changed. Each answer is a dated, hash-verified receipt, and when there were too few draws to tell, the receipt says so.
Every receipt has its own page, with its verdict, its two arms and a way to verify it yourself. How this is built describes how the tool itself is checked.
Verdict Passing
Model claude-haiku-4-5
Date 2026-09-17
Skill commit-message-conventions
Range 0.434 without, 0.822 with
the two ranges do not overlap: separated
Receipt hash fad883d1018d3a4b3c75bc60b1eaec50032256cbf69fba3299999edf7978589c
On the report that measured it, draw-to-draw spread reaches sd 0.355 on a single case (semver-hidden-breaking-change). That is the across-draw spread of the model writing a different answer, not of the scorer re-reading one, and it is wider than the scorer's.
We publish refusals instead of guesses, and we publish our own instrument defects, including what outside audits found. Read the methodology, or read what we will not claim.
See whether your custom instructions do anything, with no install.
See whether it helps before you publish, and on which model.
See which of your skills stopped separating when the model changed.
An evidence trail for the skills your agents run in production.
You need Claude Code, Node.js 22 or newer, and your project in a git repository. You do not need an API key: runs use your Claude Code plan's usage.
claude plugin marketplace add driftproofhq/driftproof
claude plugin install driftproof@driftproofhq
Each command starts the pinned runner with npx, so a first run has no extra install step. Two gaps remain, and the plugin's words cannot close them: a program in a parent folder's node_modules/.bin can run before the runner, and the line Claude runs can be altered. The safer alternative is to install the runner with npm install -g, at the version the command names, and use the installed line each command file gives. npm then starts nothing, so the first gap closes. The line Claude runs can still be altered in either form.
That installs /driftproof:start, /driftproof:init, /driftproof:run, /driftproof:blind, /driftproof:badge, /driftproof:compare, /driftproof:inspect, /driftproof:recheck and /driftproof:measure. The plugin hands your arguments to the pinned runner and writes the receipt that runner would have written; /driftproof:run spends your Claude Code subscription rather than an API key. It measures and never changes a skill's instructions. The one file it adds is a new evals/evals.json, from /driftproof:start or /driftproof:init, to a skill that has none, after you say yes in the run.
Compare and recheck need a git repository, and compare takes a rules file: CLAUDE.md, AGENTS.md, .cursor/rules/*.mdc, .cursorrules, .github/copilot-instructions.md, .github/instructions/*.instructions.md, .windsurfrules, .windsurf/rules/*.md.
A full run can take up to 120 model calls for each test task from your plan's usage. A quick check takes far fewer.
Start with a skill you already have. In Claude Code, from the folder above it, in a git repository, run /driftproof:start. Claude lists your skills and asks which to check. If the skill has no test cases, Claude reads your SKILL.md and drafts test cases with you, aimed at outcomes you would check. It writes only after you say yes, then runs a quick check and opens a results page: your first receipt. Drafted cases are labelled as drafted from the skill and approved by you. A quick check is a first look and cannot produce a verdict. For a direct measurement, use /driftproof:run path/to/your-skill. On a skill with no test cases, it offers the same draft. For a new skill, scaffold one with npx driftproof init my-skill, as the next section shows. /driftproof:start works on these rules files too: CLAUDE.md, AGENTS.md, .cursor/rules/*.mdc, .cursorrules, .github/copilot-instructions.md, .github/instructions/*.instructions.md, .windsurfrules, .windsurf/rules/*.md. Measured as always-on text. The editor's system prompt and rule selection are not reproduced.
Tested on Linux and macOS. On Windows the plugin also needs Node, npm's npx and Git for Windows on PATH, and Windows has only been checked without calling a model. The findings page has the test details.
You need Node 22 or newer and an ANTHROPIC_API_KEY. A Claude Pro or Max plan cannot be used in CI: the Action needs an API key, billed per call, so set max-usd. From a clean checkout to a receipt for a new skill, on the command line:
npx driftproof init my-skill # scaffold SKILL.md + evals/evals.json + .driftproofrc
# edit the 3 example cases so each is grounded in a claim your SKILL.md makes
export CLAUDE_PROVIDER=api
read -rsp "Anthropic API key: " ANTHROPIC_API_KEY && export ANTHROPIC_API_KEY
npx driftproof run my-skill --models claude-haiku-4-5
cat receipts/*.summary.md # read the receipt + human summary
Wire it into a repository with the GitHub Action (uses: driftproofhq/driftproof@v0.30.0): it re-runs the suite on every push, uploads the receipts, and fails the job when a case separated downward. driftproof badge turns a receipt into a badge that carries the model and the date it was measured.
Copy the pull request workflow into .github/workflows/ and add an ANTHROPIC_API_KEY repository secret. It checks changed skills and rules files with test cases, then updates a receipt table on each push. The budget is shared across files. Files without cases are named as not measured.
The step that matters, with its settings:
uses: driftproofhq/driftproof@v0.30.0
with:
api-key: ${{ secrets.ANTHROPIC_API_KEY }}
placebo: 'true'
max-usd: '20'
fail-on-regression: 'true'
fail-on-fake: 'false'
Skills need evals/evals.json in their folder. Rules files need it beside the file. The workflow pins a release. Update the pin when you choose to adopt a newer release. Forks make no model calls and get a notice in the job summary; GitHub may block their comment. The receipt links open the job's artifacts, which follow GitHub's retention period.
fail-on-regression is on by default. Turn on fail-on-fake to fail on worse than the fake, never just because it did not beat the fake. That setting needs the placebo arm enabled, as in the example. For OpenAI models, set models and openai-api-key. Only API keys are supported. A green check does not prove a file helps.
To compare a pull request's supported rules file with the version on its base branch, on the same tasks, set the Action's compare-with-base input.
Your address goes to Buttondown, our newsletter provider, and nowhere else. One email per model release. Unsubscribe any time.
Every report is also in the Atom feed.
Words with a dotted line link to the glossary, where each is explained in plain words.