Six months after your loan-approval system goes live, a regulator submits a formal request. A customer is appealing a denial dated May 12. The question is simple and lethal: why did the AI recommend denial?
If your answer is "the model decided so," you are already non-compliant. A high-risk system under the EU AI Act requires a reconstructible chain of evidence: the exact system prompt, the customer data pulled in, the model version, the guardrails that fired. Without an immutable trace of every call, the decision is unprovable, and the system cannot legally operate.
This is the audit plane: the layer where every AI decision leaves a permanent trace, sufficient to reproduce and explain it to a regulator, an auditor, or an incident review. Without it, every preceding control plane is unprovable. Guardrails that cannot be shown in a log do not exist as far as a regulator is concerned.
The Regulatory and Operational Drivers
The audit plane is driven by external law and internal incident response.
On the regulatory side, the EU AI Act (Art. 12) mandates automatic logging for the entire lifetime of a high-risk system. Art. 13 requires transparency, and Art. 86 grants the right to an explanation of an individual decision. GDPR Art. 22 reinforces this with explainability requirements for automated decisions.
On the operational side, without a trace, you cannot tell a prompt bug apart from model drift or a security attack. These are three different causes requiring three different responses. An incident review without a trace is guesswork.
The Architectural Pattern: Immutable AI Trace Log
The engineering solution is an Immutable AI Trace Log Pipeline built on OpenTelemetry GenAI Semantic Conventions. This provides end-to-end tracing with standard spans, exported to immutable storage with a defined retention policy.
The observability ecosystem for LLMs is fragmented. Langfuse, Helicone, Traceloop, and LangSmith all use incompatible proprietary formats, locking you into a vendor. GenAI Semantic Conventions, developed under the CNCF, define a shared vocabulary. A span from a LangChain agent looks exactly the same as one from a bare OpenAI call.
An honest caveat: as of mid-2026, the conventions are still in Development status. In production, teams must enable OTEL_SEMCONV_STABILITY_OPT_IN for dual emission—sending both legacy and new attributes—to survive the eventual transition to stable.
Engineering Stack
- Tracing standard: OpenTelemetry + GenAI Semantic Conventions (spans for LLM calls, retrieval, tool calls).
- Backends: Arize Phoenix, Langfuse, or LangSmith (all accept OTel).
- Transport: Kafka or NATS.
- Storage: ClickHouse for analytics and dashboards, plus an immutable WORM (Write Once Read Many) layer for legal audit.
Step 1: An End-to-End trace_id
A single ID runs the entire decision chain: the API gateway, RAG and fine-grained authorization, input guardrails, the model, tool calls, output guardrails, and the final response.
This same ID serves as the key into the PII mapping store and the budget tracker. It ties the entire system's state together for a single decision.
Step 2: What Gets Captured in the Spans
Per GenAI SemConv plus audit extensions, the spans must capture:
- The system prompt version (a hash and a link to the version control commit).
- The exact RAG chunks with their distance scores and
source_doc_id. - Raw request and response (already with PII masked).
- The arguments of every called function (tool calling) and the policy gate's verdict.
- The model/provider version, temperature, and seed.
- The guardrail verdicts (what fired, what was blocked).
Step 3: Export to Immutable Storage
The pipeline routes data from the application via the OTel SDK through the OTLP protocol to a collector, then into Kafka. From Kafka, the stream splits:
- To ClickHouse for analytics and dashboards.
- To a WORM/append-only layer for legal audit and retention.
The append-only layer is not "a database everyone agreed not to write over." It is storage with a technical impossibility of overwrite, using object-lock mechanisms.
Step 4: Reproducibility
Given a trace_id, the full input is reassembled. With a fixed model version and seed, a decision can be replayed. This is what the regulator needs: proof of what ran, what data it used, and what verdict each control produced.
Step 5: The Log's Own Privacy
The audit trail contains PII and secrets. It requires encryption, role-based access control, and a separate TTL that balances the AI Act's mandate to retain against GDPR's mandate not to over-retain.
Where It Breaks
The pipeline still fails in predictable ways. The audit plane has specific, dangerous failure modes.
LLM non-determinism. Even with a seed, reproducibility is incomplete. Providers swap models "under the hood," and sampling varies. An "explanation" produced by replaying a trace is a reconstruction of the decision's context, not a guarantee of literal causal replication.
Post-hoc rationalization. Chain-of-thought logged from the model is what the model wrote, not necessarily why it decided that way. Do not sell CoT to a regulator as a causal mechanism. It is a artifact of the generation process, not the generation's driver.
Volume and cost. A full trace with RAG chunks runs into terabytes. Retention under Art. 12 versus the ClickHouse budget versus GDPR data minimization is a three-way conflict. It is resolved by sampling detail level: full trace for high-risk decisions, truncated for the rest.
The log as an attack surface. An audit trail containing PII and secrets is a prime target. A leak of the audit log is worse than the original data breach. Protect the log as part of the security perimeter.
The standard isn't stable yet. SemConv is in Development. Attributes change, which is why dual emission during the transition period is mandatory for production systems.
Maturity Checklist
- L1: Raw request/response and model version are logged.
- L2: An OTel trace (SemConv) with chunks, prompt version, and tool arguments stored in append-only storage.
- L3: Decision replay by
trace_id, retention aligned with the AI Act, encryption and RBAC on the audit trail, dual emission during the standard's transition period.
Standards and Mapping
- EU AI Act: Art. 12 (record-keeping), Art. 13 (transparency), Art. 86 (explanation).
- ISO/IEC 42001: logging, monitoring, incident management.
- NIST AI RMF: Measure/Manage — traceability.
- GDPR: Art. 22.
When the regulator asks for the May 12 decision, you do not open the model and hope for the best. You pull the trace_id, reconstruct the exact prompt, the retrieved chunks, the guardrail verdicts, and the model version, and you hand over the evidence. If you cannot do that, the system you built is a liability.