Six months after your loan-approval system goes live, a regulator submits a formal request. A customer is appealing a denial dated May 12. The question is simple and lethal: why did the AI recommend denial?
If your answer is "the model decided so," you are already non-compliant. A high-risk system under the EU AI Act requires a reconstructible chain of evidence: the exact system prompt, the customer data pulled in, the model version, the guardrails that fired. Without an immutable trace of every call, the decision is unprovable, and the system cannot legally operate.