500xEngineer.

All postsSHORT5 min read

Claude Code Verify Skill: Who Checks Claude?

The Claude Code team says Claude writes and reviews about 90% of its code. Its post explains verification, human handoffs and protected checks.

The Claude Code team says Claude writes and reviews about 90% of the team's own code. Who checks Claude? Its answer is verification: running the change, checking the evidence and bringing in a human when checks keep failing.

Video not loading? Watch the Short on YouTube

What the team said

Anthropic published Addy Osmani's post, How the Claude Code team thinks about verification, on November 16, 2026. These are the Claude Code team's reported practices and figures for its own work.

The team says verification loops let it delegate more work. Per the post, without those loops you can only run 1 or 2 Claudes at a time because you have to check everything yourself. With them, the post says, you can run dozens or hundreds in parallel. Those are the team's claims, with no outside measurement supplied.

The post also reports that in Anthropic's internal testing on structured file generation, a separate grader checking Claude's work against a rubric improved task success by up to 10 points over a standard prompting loop. The post says the largest gains were on the hardest problems. That reported test concerns structured file generation with a separate grader.

Verification means running the change

The team's post says: "Verification means running the change." Its advice is to run the app end to end, including trying an iOS change in a simulator.

The post recommends the Claude Chrome extension for web changes, an iOS or Android MCP for mobile, computer use for desktop, and running the service locally for backend changes.

To make those instructions repeatable, the post tells readers to create a skill called verify in the repo's .claude/skills/ folder, close to the relevant code. Add as many as needed. The post says Claude knows to use the verify skill before committing changes. Readers still have to create that skill for their app and make it fast and reliable.

The 3 test types the post recommends

The team's post recommends end-to-end tests, property tests and differential tests. If you have to focus on one type, it recommends end-to-end.

The post describes end-to-end tests through real user flows: adding an item to a cart, checking out and reaching a confirmation page. It says unit tests are faster and less flaky, but they can pass while a feature is broken.

Per the team, Claude runs tests, the build and the linter before opening a pull request. The post suggests enforcing tests with a hook when Claude finishes, or using a /goal of a passing test suite.

3 rounds, then a human

The team says Claude runs the checks and tries to fix failures. Per its post, if a check still fails after 3 rounds, Claude stops and brings in a human.

The team's post also describes how it handles flaky tests. Claude can't skip them, add sleeps or keep retrying until they pass. If it can't find the root cause, the post says it deletes the test and records why. That's the team's stated handling of flaky tests, alongside its human handoff rule.

Checks the change can't edit

The team's rule is that a change can't modify the checks judging it. Per the post, Code Review reads REVIEW.md from the base or default branch, never from the pull request's changes.

The team describes tuning a reviewer through REVIEW.md, which says what deserves attention in that code. Code owners write the checks.

Per the post, patterns from human review comments and pull request reverts must appear in at least 2 pull requests and be identifiable from a diff before becoming potential rules. The post says 3 agents try to disprove those rules, and a human reviews and approves them in a pull request.

The team's internal checklist and commit stamps

The team says it maintains a 39-item reliability checklist. Per the post, each item has a trigger visible in a diff, a check, and the incidents or pull requests behind it. Claude checks only matching items, usually 3 to 10, according to the team.

In its own repos, the team says it stamps each commit with how it was verified. The stamp can say none, letting the team find unverified commits.

The team also describes an internal commit gate that can block a commit when the session lacks /code-review or /verify. Per the post, it has off, observe and enforce modes. If commits have been blocked twice in a session, the team says the next commit is allowed through so Claude can continue.

The checklist, stamps and gate described here are the team's internal practices. The reader-facing instruction in the post is to create a verify skill for your own repo.

The catch

Anthropic sells Claude Code and published this account of its team's work. The post supplies no independent measurement of the Claude Code team's about 90% figure, defect rates, bugs caught or time saved.

Humans still write the checks, take over after 3 failed rounds and approve review rules, according to the team. The post itself says "Claude Code isn't perfect". The internal gate's allowance after repeated blocks matters when you're deciding how much oversight your own setup needs.

The post gives no price for this setup.

How to set up your own verify skill

Following the team's post:

  • Create a skill called verify in .claude/skills/, close to the relevant code. Describe how to start the simulator or run the app.
  • Have it run a real user flow end to end. Use the web, mobile, desktop or backend approach the post describes for your app.
  • Make the skill fast and reliable. The post treats it as your smoke test.
  • Run tests, the build and the linter before opening a pull request. The post suggests a hook to run tests when Claude finishes, or a /goal of a passing test suite.
  • Follow the team's handoff rule: Claude tries to fix failing checks, then stops for a human after 3 failed rounds.
  • Keep judging checks beyond the change's control. The team's example is reading REVIEW.md from the base or default branch.

The team's broader advice is to define what done looks like and review the evidence. That's the work behind its delegation claim.

The builder angle: my opinion

Sell small dev teams a verify skill setup that boots their app and walks a real user flow. Include a stop rule that hands off to a human after 3 failed rounds, plus judging checks the change can't edit.

The deliverable is a skill for their repo and app, following the practices the team's post describes. This is my opinion, with no income guaranteed.

Daily email

Get each breakdown in your inbox.

One email each evening, right after the Short goes live: the AI tool it covers, the full write-up with sources, and how to get paid for it. Free, and you can unsubscribe anytime.

Read past emails on Substack

Sources

All postsWatch on YouTube