As of May 2026, Anthropic says Claude authored more than 80% of the code they merge into their production codebase (When AI builds itself). The number is public and it's Anthropic talking about Anthropic — not a verified fact about your organization, and it shouldn't be treated as one. But as a market signal it's worth sitting with: you probably have Claude Code licenses too, and your team is still using it as a smart autocomplete — finish a line, generate a test, speed up the boring part.
The gap isn't budget. It's what happens around the tool after the money is spent. The Anthropic number is worth reading as an indicator: wherever that ratio is real, organizational work preceded it — work that most companies holding the same licenses simply skipped. Buying keys and seats is not enablement. Without preparation for the AI-shaped way of working, the tool stays exactly what the team made of it: a well-regarded autocomplete.
Why a "what is RAG" workshop fixes nothing
Until that preparation exists, the team has no answer to three plain questions. Which tasks are actually worth handing to AI, and which aren't. How to review what it returns, when review used to mean reviewing human-written code. How to wire that review loop into the normal SDLC so output quality doesn't quietly drift downward over the next two or three months.
Every unanswered question is a cost the company is already paying: engineer hours spent decoding AI output nobody fully trusts, the risk of missing a regression a human reviewer used to catch, and the nagging sense that an expensive tool somehow isn't delivering the speed it promised. A "what is RAG" workshop doesn't lower that cost. Neither does a Zoom demo — it shows what the model can do, but changes nothing about how the team works with it tomorrow, on a real ticket.
Three things that have to be ready before you start
For AI to be embedded with intent rather than in scattered pockets, three things need to be true at the same time.
Processes are decomposed for the AI-shaped format — a task is broken down so it's clear which slice goes to the agent and which stays with a human. Data is accessible and structured so an agent can read and act on it, instead of hitting closed formats and manual exports. And review gates are defined in advance, before the agent produces anything — not invented after the fact, once the output is already sitting in a pull request and someone has to figure out, on the spot, how to judge it.
That last point isn't a nice-to-have. Defining the review gate ahead of time means deciding what gets checked and against which criterion before you open a diff — instead of reading it line by line and hoping to catch the problem by eye. That's the exact trap that keeps engineers stuck in review theater instead of doing the engineering work of writing acceptance criteria: reading AI-generated code line by line looks like effort, but it doesn't substitute for a criterion defined before the code existed.
Without all three, AI has nothing to work on and nowhere to work at scale — no matter how capable the model behind the license is.
Decomposing for the AI-shaped format is a specific split by risk type, not the generic "break tasks into smaller pieces" advice: hand the agent work with a verifiable outcome (a data-schema migration, tests generated against acceptance criteria that already exist, refactoring against a fixed pattern), and keep with a human the work where the cost of a mistake is high and the acceptance criterion can't be formalized ahead of time (architectural forks, integrations with external systems with no explicit contract). Without that split, you get what's happening at most companies with licenses already purchased: the agent gets handed everything, and someone reads through the entire output by hand afterward — because nobody decided in advance which slice could be trusted by default.
Where enablement stops being theory
The gap closes when the practice starts living inside the normal delivery flow, not through training that sits apart from the work. Real backlog tickets become the training material instead of synthetic examples. Playbooks for reviewing AI output live in the same PR flow as the rest of the code — a wiki page nobody opens twice usually doesn't cut it. Evals are wired to releases. And the practice has an owner: one named person with authority and a KPI on AI-output quality. Diffuse collective responsibility doesn't reach that level of specificity.
That's what it looks like in an enterprise setting in practice: the work produces real artifacts that stay in production after the session ends. A conversational SQL bot in front of a data lake. A next-best-offer ML model. A geo-resolution proxy microservice and a timestamp tracker, both written through AI-assisted coding during a training session with the team — dozens of sessions, 100–200 people each, producing shipped code instead of slides.
The difference from ordinary corporate training isn't the delivery format, it's the subject matter. A regular training course teaches a concept on a synthetic example that gets forgotten within a week because there's nowhere to apply it. Here, the training session itself produces a production artifact: participants leave not with a completion certificate but with a working service their own team is operating the next day. That changes the motivation to actually absorb the material — a mistake in the exercise isn't abstract, it shows up in production within a day.
The mechanics of review: three layers instead of one final click
Reviewing AI output in this setup isn't a single final "looks fine" check. It's three layers, each with its own criterion and its own moment in the release cycle.
Layer 1 — Regression. A fixed set of reference cases with known expected outputs: edge cases from past incidents, the same cases that were used to approve the feature in the demo (run those too, but not only those), and at least one adversarial input per output type — prompt injection, empty input, malformed input. Any new failure blocks release. The field that matters most in each case is origin: it has to point to a real incident, not a slide from a pitch deck.
id: rag-stale-corpus
origin: incident-2026-03
input:
query: "current rate for tariff X"
expected:
must_contain:
- "current document"
must_not_contain:
- "archived"
pass_criteria:
- no_archived_docs
- answer_groundedLayer 2 — Distribution check. For RAG and classifiers: 20–50 fresh real inputs run against the new version, compared not to an "ideal" but as a diff against the previous release's snapshot — output length and format distribution, changes in retrieved chunks, confidence shifting toward the edges. A shift to the edges — the model is either certain about everything or nothing — is the fragility signal, not noise.
Layer 3 — Human spot-check. Mandatory before the first production deploy of a new output type: a domain expert reads 10–20 real outputs against specific questions — is there data the model shouldn't have access to; would an expert call this correct; is there adversarial exploitation an automated test wouldn't catch.
Owner. One person signs off across all three layers. If you can't name that person before release, that's already a risk signal — waiting won't make it go away. The absence of a named owner is what turns "looked fine in staging" into a habit of delegating accountability to no one in particular.
The mechanics and a working harness for the first two layers are covered in detail in eval as a release criterion against demo-grade, which also ships a minimal harness with explicit drift thresholds.
The demo-grade trap
On paper, three layers look like sufficient insurance. But this is exactly where the main trap lives — demo-grade eval: a feature gets approved on the same inputs it was pitched to an investor or client with. Tests pass, the demo shines, and in production, across the real variety of inputs, the model behaves differently — and nobody can point to where that should have been caught.
Three concrete failure modes account for most of this gap in practice: a RAG system answers correctly on a curated staging corpus and hallucinates on the full corpus with stale entries; a classifier silently degrades after an upstream input-schema change, with no one notified; AI-generated code passes every automated test but introduces a security pattern those tests never covered. Three different stacks — one shared root cause: the eval dataset matched the demo dataset.
There are two more systemic failures adjacent to demo-grade that are worth naming separately. First, pass criteria get written after the run, based on what came out, instead of being fixed beforehand — "looks reasonable" instead of a threshold set in advance. Second, the snapshot used for the distribution diff gets quietly updated at the moment of degradation, instead of someone investigating the shift; bad behavior silently becomes the new baseline, and the next release's diff won't catch it anymore.
What separates enablement from a demo
The most unexpected part here isn't the technology or the review method — it's what the team actually remembers. People absorb how to work with AI at the moment a training session leaves behind working code that actually ships, not from a lecture or a polished demo. The line between real enablement and its imitation comes down to one question: what happens to the code produced after the session ends — does it go to production, or does it dissolve along with the slides.
The same question holds at the level of the whole SDLC, not just the review moment: if decomposition, data access, and review are built into the normal delivery flow from day one, AI covers the full cycle, not just writing code — from the clarify dialog with a stakeholder to production monitoring. That embeddedness separates enablement from a one-off demonstration of what the model can do.
Why this works because of the enterprise
A large organization has a perfectly rational fear: breaking processes that already work for the sake of a tool that hasn't yet proven it's predictable. The actual ask here isn't "roll out AI faster" — it's predictable delivery of AI value inside existing governance constraints, without risking what's already stable. Enablement built into the real delivery flow — with an owner, playbooks, and evals wired to releases — answers that ask: it doesn't promise a revolution in one session, but it also doesn't force a choice between AI speed and the control an enterprise isn't willing to give up.
Repo with the minimal harness for the first two layers: github.com/dobryakov/eval-harness.
If you already have Claude Code licenses purchased and your production output is still at the level of interesting demos, the problem isn't the tool or the model behind the subscription — it's the practice that should have existed around it before the first agent ever touched real code.