Is the gap your skill makes real, or noise? Did it hold on the last model release?

A SKILL.md teaches an AI coding agent how you like things done. A skill's own tests passing is one answer. Driftproof asks two more: whether the scores with the skill and without it separate beyond their spread, and whether that held when the model changed. Each answer is a dated, hash-verified receipt, and when there were too few draws to tell, the receipt says so.

Install in Claude Code

claude plugin marketplace add driftproofhq/driftproof claude plugin install driftproof@driftproofhq

Then run /driftproof:run on a skill in your repository. For CI, npx and the GitHub Action are further down.

See a receipt Read the findings Read the latest report

A receipt, in full

Verdict Passing

Model claude-haiku-4-5

Date 2026-09-17

Skill commit-message-conventions

Range 0.434 without, 0.822 with

Receipt hash fad883d1018d3a4b3c75bc60b1eaec50032256cbf69fba3299999edf7978589c

What a receipt can say

Passing. A case separated upward under the band rule: the two ranges do not overlap and the move clears the effect floor. That is a separation detected under the rule, not proof that the skill moved the score.
Regressed. A case separated downward under the same rule.
No separation detected. The ranges overlap at this sample size, which is not evidence that nothing changed.
Not enough draws. Not enough draws to conclude at this effect floor. The receipt prints how many draws per arm would have been needed, or that no number of draws reaches the floor at the spreads it measured.
Not measured. The receipt carries no verdict, and its page names every reason, such as a run that does not record that a model answered it.

Every receipt has its own page, with its verdict, its two arms and a way to verify it yourself. How this is built describes how the tool itself is checked.

How it works

Run your skill's tests twice: with the skill, and without it.
Draw each answer several times and score each draw several times, so every result is a range, not one fragile number.
Call it a separation only when the ranges don't overlap and the move clears the floor. When the ranges overlap, say no separation detected at this sample size; when the draws were too few to tell, say so.

That range is a band. The dated file that records all of it is a receipt. What changes between two receipts is drift.

Two horizontal ranges, drawn apart. — Separated The two ranges do not overlap, and the move clears the floor. A separation is detected under the rule, which is not proof that the skill moved.

Two horizontal ranges, drawn crossing each other. — Overlapping The ranges share space. We say no separation detected at this sample size, and we say it plainly.

Two horizontal ranges, drawn with one hatched because nothing was measured. — Refused The run could not stand behind a number, so it reports no result instead of guessing one.

The same two ranges, drawn from the bundled example receipt.

Move the floor and change what counts as a band to see the verdict change. This is the bundled example receipt's own per-case data.

Skills don't break. Models move.

11reports published

134receipts published

9models measured on

867eval cases scored

3refusals published

119runs with no separation detected at their sample size

On the report that measured it, draw-to-draw spread reaches sd 0.355 on a single case (semver-hidden-breaking-change). That is the across-draw spread of the model writing a different answer, not of the scorer re-reading one, and it is wider than the scorer's.

We publish refusals instead of guesses, and we publish our own instrument defects, including what outside audits found. Read the methodology, or read what we will not claim.

Who it's for

Skill authors

See whether it helps before you publish, and on which model.

Catalog maintainers

See which of your skills stopped separating when the model changed.

Platform teams

An evidence trail for the skills your agents run in production.

Install in Claude Code

claude plugin marketplace add driftproofhq/driftproof

claude plugin install driftproof@driftproofhq

That installs /driftproof:init, /driftproof:run and /driftproof:badge. The plugin hands your arguments to the pinned runner and writes the receipt that runner would have written; /driftproof:run spends your Claude Code subscription rather than an API key, and it never edits a skill.

Driftproof is tested on Linux and macOS. In an outside retest on macOS the shipped gate failed only in the publishing helper, which needs GNU realpath -m. Windows is untested: from Node's source the plugin would not find npx there, but that failure has never been observed. A CI matrix across Linux, macOS and Windows is planned, and the findings page has the figures.

In CI: npx and the GitHub Action

You need Node 22 or newer and an ANTHROPIC_API_KEY. From a clean checkout to a receipt for your own skill:

npx driftproof init my-skill          # scaffold SKILL.md + evals/evals.json + .driftproofrc
# edit the 3 example cases so each is grounded in a claim your SKILL.md makes
export CLAUDE_PROVIDER=api
read -rsp "Anthropic API key: " ANTHROPIC_API_KEY && export ANTHROPIC_API_KEY
npx driftproof run my-skill --models claude-haiku-4-5
cat receipts/*.summary.md              # read the receipt + human summary

Wire it into a repository with the GitHub Action (uses: driftproofhq/driftproof@v0.11.2): it re-runs the suite on every push, uploads the receipts, and fails the job when a case separated downward. driftproof badge turns a receipt into a badge that carries the model and the date it was measured.

driftproof badge: passing on claude-haiku-4-5

Latest report

Release drift report

Report 011: Claude Opus 5.5 on release day, three skills

claude-opus-5 claude-opus-5-5

2026-09-23

All 11 reports

Writing: Three model releases later: what actually happens to agent skills, the launch essay, revised to read all eleven reports together on what a moving model does to encoded expertise.

Get the next one

Your address goes to Buttondown, our newsletter provider, and nowhere else. One email per model release. Unsubscribe any time.

Every report is also in the Atom feed.

Roadmap

Next: activation checks.
Planned, not scheduled: a sandboxed execution harness for tool-execution skills (document renderers, diagram and asset generators), which Report 001 scopes out.