Six months after your loan-approval system goes live, a regulator submits a formal request. A customer is appealing a denial dated May 12. The question is simple and lethal: why did the AI recommend denial?<\/p>\n
If your answer is "the model decided so," you are already non-compliant. A high-risk system under the EU AI Act requires a reconstructible chain of evidence: the exact system prompt, the customer data pulled in, the model version, the guardrails that fired. Without an immutable trace of every call, the decision is unprovable, and the system cannot legally operate.<\/p>\n
<\/p>\n
This is the audit plane: the layer where every AI decision leaves a permanent trace, sufficient to reproduce and explain it to a regulator, an auditor, or an incident review. Without it, every preceding control plane is unprovable. Guardrails that cannot be shown in a log do not exist as far as a regulator is concerned.<\/p>\n
The audit plane is driven by external law and internal incident response.<\/p>\n
On the regulatory side, the EU AI Act (Art. 12) mandates automatic logging for the entire lifetime of a high-risk system. Art. 13 requires transparency, and Art. 86 grants the right to an explanation of an individual decision. GDPR Art. 22 reinforces this with explainability requirements for automated decisions.<\/p>\n
On the operational side, without a trace, you cannot tell a prompt bug apart from model drift or a security attack. These are three different causes requiring three different responses. An incident review without a trace is guesswork.<\/p>\n
The engineering solution is an Immutable AI Trace Log Pipeline built on OpenTelemetry GenAI Semantic Conventions. This provides end-to-end tracing with standard spans, exported to immutable storage with a defined retention policy.<\/p>\n
The observability ecosystem for LLMs is fragmented. Langfuse, Helicone, Traceloop, and LangSmith all use incompatible proprietary formats, locking you into a vendor. GenAI Semantic Conventions, developed under the CNCF, define a shared vocabulary. A span from a LangChain agent looks exactly the same as one from a bare OpenAI call.<\/p>\n
An honest caveat: as of mid-2026, the conventions are still in Development status. In production, teams must enable A single ID runs the entire decision chain: the API gateway, RAG and fine-grained authorization, input guardrails, the model, tool calls, output guardrails, and the final response.<\/p>\n This same ID serves as the key into the PII mapping store and the budget tracker. It ties the entire system's state together for a single decision.<\/p>\n Per GenAI SemConv plus audit extensions, the spans must capture:<\/p>\n The pipeline routes data from the application via the OTel SDK through the OTLP protocol to a collector, then into Kafka. From Kafka, the stream splits:<\/p>\n The append-only layer is not "a database everyone agreed not to write over." It is storage with a technical impossibility of overwrite, using object-lock mechanisms.<\/p>\n Given a The audit trail contains PII and secrets. It requires encryption, role-based access control, and a separate TTL that balances the AI Act's mandate to retain against GDPR's mandate not to over-retain.<\/p>\n The pipeline still fails in predictable ways. The audit plane has specific, dangerous failure modes.<\/p>\n LLM non-determinism.<\/strong> Even with a seed, reproducibility is incomplete. Providers swap models "under the hood," and sampling varies. An "explanation" produced by replaying a trace is a reconstruction of the decision's context, not a guarantee of literal causal replication.<\/p>\n Post-hoc rationalization.<\/strong> Chain-of-thought logged from the model is what the model wrote, not necessarily why it decided that way. Do not sell CoT to a regulator as a causal mechanism. It is a artifact of the generation process, not the generation's driver.<\/p>\n Volume and cost.<\/strong> A full trace with RAG chunks runs into terabytes. Retention under Art. 12 versus the ClickHouse budget versus GDPR data minimization is a three-way conflict. It is resolved by sampling detail level: full trace for high-risk decisions, truncated for the rest.<\/p>\n The log as an attack surface.<\/strong> An audit trail containing PII and secrets is a prime target. A leak of the audit log is worse than the original data breach. Protect the log as part of the security perimeter.<\/p>\n The standard isn't stable yet.<\/strong> SemConv is in Development. Attributes change, which is why dual emission during the transition period is mandatory for production systems.<\/p>\n When the regulator asks for the May 12 decision, you do not open the model and hope for the best. You pull the Six months after launch, a regulator asks why your AI denied a loan. Without an immutable trace of every call, you have no answer \u2014 and a high-risk system that is non-compliant.<\/p>\n","protected":false},"author":0,"featured_media":175,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[147],"tags":[164,150,161,153,162,163],"class_list":["post-176","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-governance","tag-ai-trace","tag-ai-governance","tag-auditability","tag-eu-ai-act","tag-opentelemetry","tag-reproducibility"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\tOTEL_SEMCONV_STABILITY_OPT_IN<\/code> for dual emission\u2014sending both legacy and new attributes\u2014to survive the eventual transition to stable.<\/p>\nEngineering Stack<\/h2>\n
\n
Step 1: An End-to-End
trace_id<\/code><\/h2>\nStep 2: What Gets Captured in the Spans<\/h2>\n
\n
source_doc_id<\/code>.<\/li>\nStep 3: Export to Immutable Storage<\/h2>\n
\n
Step 4: Reproducibility<\/h2>\n
trace_id<\/code>, the full input is reassembled. With a fixed model version and seed, a decision can be replayed. This is what the regulator needs: proof of what ran, what data it used, and what verdict each control produced.<\/p>\nStep 5: The Log's Own Privacy<\/h2>\n
Where It Breaks<\/h2>\n
Maturity Checklist<\/h2>\n
\n
trace_id<\/code>, retention aligned with the AI Act, encryption and RBAC on the audit trail, dual emission during the standard's transition period.<\/li>\n<\/ul>\nStandards and Mapping<\/h2>\n
\n
trace_id<\/code>, reconstruct the exact prompt, the retrieved chunks, the guardrail verdicts, and the model version, and you hand over the evidence. If you cannot do that, the system you built is a liability.<\/p>\n","protected":false},"excerpt":{"rendered":"