Tag: demo-grade

Buying Claude Licenses Is Not Enablement: What Has to Be Ready Before You Start

As of May 2026, Anthropic says Claude authored more than 80% of the code they merge into production — their figure, not a fact about your org. What it points to is real: buying licenses is not enablement. What has to be ready before you start, and how a three-layer review catches demo-grade output before it reaches production.

As of May 2026, Anthropic says Claude authored more than 80% of the code they merge into their production codebase (When AI builds itself). The number is public and it's Anthropic talking about Anthropic — not a verified fact about your organization, and it shouldn't be treated as one. But as a market signal it's worth sitting with: you probably have Claude Code licenses too, and your team is still using it as a smart autocomplete — finish a line, generate a test, speed up the boring part.

Continue reading “Buying Claude Licenses Is Not Enablement: What Has to Be Ready Before You Start”

How to make eval a release criterion for AI features instead of a demo-grade approval

Demo-grade eval greenlights a feature on the same examples as the pitch. Production then shows the long tail. Three layers — incident-born regression, distribution diff vs the last snapshot, human spot-check — plus one named owner. Minimal harness: github.com/dobryakov/eval-harness.

Treat AI-output quality as a release criterion: a fixed regression set built from past incidents, a distribution diff against the previous release snapshot (20–50 real inputs), a human spot-check before the first production deploy of a new output type — and one person who signs off. Below: the demo-grade anti-pattern, the three-layer method, and a minimal harness that fails CI with a readable exit code.

Continue reading “How to make eval a release criterion for AI features instead of a demo-grade approval”