Why Documentation Still Matters When the Agent Reads the Code Directly

Code tells an AI agent how the system works today. Specs tell it how the system should work. Without the second layer, the agent cannot tell a bug from a feature.

Why Documentation Still Matters When the Agent Reads the Code Directly

Suppose an agent can load the whole monorepo into context and generate a correct-looking function. The moment developers see this, some teams conclude: if the agent can read all the code at once, documentation is obsolete. Specifications, architecture descriptions, technical requirements — unnecessary process overhead. Why write text when the agent reads the code directly and sees ground truth?

Code is evidence of the current implementation, not proof of the intended behavior.

Why Code Alone Is an Unreliable Source of Requirements

The conclusion that "code is the Single Source of Truth" assumes development is mostly source-code editing: the belief that engineering is reducible to writing source files. Look at the SDLC from the perspective of someone focused only on implementation, and yes, code seems like the most visible artifact. But the moment you embed an autonomous or semi-autonomous agent into the engineering process, the absence of a formalized system description turns its work into a closed loop.

The distinction is between implementation and intent. Source code describes mechanics — how the system works right now. Documentation and specs describe intent — how the system should work and which business constraints it satisfies. Code shows current behavior; specifications record intended behavior, constraints, and reasons. These are two distinct layers, and the second does not derive from the first.

When an agent designs a solution, refactors, or writes tests relying exclusively on code, it may treat existing behavior as required behavior. If the code contains a logic error, a suboptimal workaround, or a hidden bug, the agent takes it for intentional behavior. Without an external source of context, it has no reliable basis for distinguishing a feature from a bug. Everything written is the benchmark.

The result can be unnecessary rewrites, tests that encode incorrect behavior, and postmortems that misattribute the failure to the model: "AI can't handle architecture." In that case, the agent lacked a recorded target state to compare the code against.

Task Definition: Requirement Validation Fails When Constraints Are Missing

Before implementation, the task needs to define expected behavior and constraints. In a normal process, an engineer reads a ticket, cross-references it with the architecture, and asks clarifying questions. When that same task reaches an agent, it should validate the requirement before generating a pull request. It must first understand what is permitted, and only then modify code.

Remove the knowledge base, and this phase produces failures that look like hallucinations but are actually missing requirement context.

One common failure is violating an implicit external contract. The agent changes an API response format or payment processing logic because the code doesn't show that an external legacy system expects a strictly defined string format. The constraint doesn't exist in the code — it lives in an agreement nobody wrote down.

Another failure is producing a solution that conflicts with the intended architecture. The agent proposes a locally working solution that does not follow the system architecture: it sticks a direct database query where the guideline mandates going through an event bus. The change may pass local tests while violating a system-level architectural rule.

Up-to-date documentation — OpenAPI, ADRs, C4 models, or simply structured Markdown in the repository — lets the agent check the task against recorded constraints before editing code. It cross-references the new requirement against existing Architecture Decision Records and business rules. Finds a conflict — rejects the task or requests clarification from the architect before hundreds of lines get rewritten. Along the way, documentation tells the agent which modules the change touches: context gets localized, token consumption drops, irrelevant files can be excluded from the context window.

Tests Can Encode Existing Bugs When No Specification Exists

The most damaging failure mode in the "code instead of documentation" idea surfaces not at task definition, but at automated testing and autofix. Here the model can validate code against the same code-derived behavior.

Give the agent a task to cover a module with integration tests or verify it after a refactor. Compare two scenarios.

Scenario A — no documentation. The agent analyzes the calculate_discount() function. Due to a previous developer's error, the discount is applied twice under a certain condition. The agent considers this normal behavior — that's how it's written in the code. It writes a test that asserts and seals the bug: assert calculate_discount() == double_discount. Tests go green. The system is broken, but the automation reports perfect coverage.

Scenario B — with a specification. The agent reads the business requirement: "Discount cannot exceed 15% and does not stack with promo codes." It compares the requirement against the calculate_discount() code. It detects a discrepancy between intent and implementation. Then it either localizes and fixes the bug, or generates a failing test that highlights the problem for the engineer.

This distinction is central to the argument. If the agent relies solely on code, any bug becomes unverifiable. Worse — the agent starts asserting that the erroneous behavior is the benchmark. It can encode the erroneous behavior in a test, making future changes preserve that behavior. The bug becomes part of the regression suite.

You can only validate an implementation against an intent that lives outside that implementation. Checking implementation only against observed implementation cannot prove that the behavior matches the intended requirement. A specification is not only documentation for humans; it's the only external reference point by which "what is written" gets separated from "what should be."

Documentation Should Be Machine-Checkable and Kept Near the Code

For agents to use documentation reliably, it must be structured and kept current. Outdated, unstructured wiki pages are equally useless to humans and LLMs. The barrier against errors isn't unvalidated prose disconnected from the codebase — the useful forms are executable specifications, ADRs, and CI-validated documentation.

Executable specs. OpenAPI and Swagger, AsyncAPI, JSON Schema, Protobuf. The agent uses them as machine-checkable constraints: CI or generation checks can reject code that violates the schema.

Architecture Decision Records. Short Markdown files directly in the repository, like docs/adr/0004-use-nats-for-messaging.md, explaining why a particular decision was made. This helps prevent the agent from replacing an intentional architectural decision with a locally simpler implementation: it sees not just the rule, but the reason for the rule.

CI-validated documentation. Documentation that CI/CD validates through CI checks, contract tests, or schema validation. The spec diverges from the API — the build fails. The same principle works at the behavior layer: an eval set becomes a release gate against solutions that pass examples but fail required behavior checks, cutting off changes that pass local checks but fail integration or release criteria before production. One repository layout is: for example: specify the requirement, clarify ambiguities, create an implementation plan, then split it into tasks, a .specify/ directory with repository-level rules and durable decisions (templates plus the memory of durable decisions), and a separate eval-harness as the release gate. Example repositories: github.com/dobryakov/ytrader-bybit and github.com/dobryakov/eval-harness.

In this form, documentation becomes an input the agent can use to check requirements and constrain code changes. A useful validation model separates intent, implementation, and behavior, where each level checks its own question. Specs and requirements validate intent — that's task definition. Source code validates implementation — that's writing code. Automated tests validate behavior — that's E2E and integration. Without recorded intent, code review and tests can only confirm consistency with the current implementation, not correctness against requirements.

Specifications Keep Humans Responsible for Intent

Removing documentation may reduce short-term writing effort, but it increases the risk of incorrect automated changes. Agents can analyze large codebases effectively. But code without a specification doesn't answer the core verification question: is what's written here actually correct?

It's through specifications, constraints, and architectural decisions that a human stays the controller and task-setter, while the agent operates within constraints set by humans. Remove that layer, and the agent may reinforce incorrect assumptions across iterations — incorrect behavior preserved by passing tests and automated approvals. The better question is: who defines intended behavior, and which artifact is used to verify the implementation?

Leave a Reply

Your email address will not be published. Required fields are marked *