Chapter 3. Burden I: risk management system, data, documentation

The situation at Kompas

Kompas is high-risk (ch. 2). Now the list of duties of a high-risk-system provider (Art. 8–17) unfolds before the team: eight items, each with sub-points. The first reaction — "we'll break it into tasks, hire someone, close it by the deadline." And then the main property of this chapter surfaces: three of the duties cannot be closed after the fact, because they describe not a document but the traces of decisions that have already been made or will still be made along the way through development. They can't be assembled a week before the audit — they can only start being kept now.

Moreover, part of the requirements hits decisions made even before the team ever heard about the AI Act — decisions about the data on which the scoring logic is tuned. This chapter is about the three obligations that set the pace for the whole project: the risk management system, data governance and technical documentation.

What the rule says

Risk Management System (Art. 9). Not a one-off artifact, but a continuous iterative process across the whole lifecycle of the system: systematically identify and analyze risks to health, safety and the fundamental rights of people, evaluate them, adopt mitigation measures, test residual risk. For Kompas the key phrase is "fundamental rights": the risk here is not a technical failure but discrimination against a candidate, an opaque rejection, systemic bias against a group. The RMS must live throughout operation, not be closed with a signature at the start.

Data & data governance (Art. 10). Requirements for training, validation and test data sets: relevance, sufficient representativeness, freedom from errors as far as possible, completeness for the intended purpose. Separately — the duty to examine possible biases that affect health, safety or lead to discrimination. For hiring this is the center of gravity: if the data on which the system learned to distinguish "strong" from "weak" candidates reflects historical skews (who was hired before), the system will entrench and scale them.

Technical documentation (Art. 11 + Annex IV). A technical file drawn up before placing on the market and kept current: description of the system and its purpose, architecture, data used, accuracy metrics, risk assessment, human oversight measures. It is the primary artifact you present to the regulator and the auditor. The crucial property: the documentation isn't written after the fact as a report — it records design decisions as they are made. A technical file reconstructed from memory before an inspection is a fiction, and an experienced auditor sees it.

How it lands on the product

Kompas sets up its RMS as a process, not a PDF: a risk registry (discrimination, quality drift, erroneous rejections) tied to releases — every material change to the system goes through a risk re-assessment. This maps onto the principle of explicit state transitions: the risk status is an external record in the registry, not an "if" in someone's head.

Data governance: Kompas sets up a passport for the scoring data — where the data comes from, how representative it is by demographics, which bias tests were run and with what result. Here a specific issue surfaces: Kompas's logic is partly determined by someone else's foundation model, whose training data the team doesn't control. What part of the Art. 10 requirements is passed up to the model provider, and what stays with Kompas — is the subject of chapters 5 and 7; but the gap needs to be recorded already now.

The technical file is set up from day one along the Annex IV structure and grows together with the product. Not a separate "by the deadline" document, but a living artifact into which decisions flow.

Where it breaks

Someone else's model breaks Art. 10. Kompas's provider does not control the foundation model's training data — but is formally responsible for the quality of its own system's data. How to meet representativeness and bias requirements for something trained not by you — is a question without a clean answer; a partial solution (documentation from the GPAI provider) is in ch. 7, but the zone stays murky.

There are no universal thresholds. "Sufficient representativeness" and "acceptable bias" have no numeric boundary in the law. What counts as sufficient is an engineering and ethical decision you'll have to defend, not one you look up in a table.

RMS easily degenerates into theater. A risk registry set up for show and never touched is the same PDF, just in a different file. A living process differs from a dead one in that a change to the product actually triggers a re-assessment, rather than passing unnoticed.

What to do as engineer/product

Set up the technical file and the risk registry now, not before the audit, and tie their updates to the release process — that way the documentation records decisions at the moment they're made, rather than being reconstructed after the fact. Explicit statuses instead of encapsulated logic: risk, data version, bias-test result — observable records.

For someone else's model — demand from the provider the documentation needed for Annex IV and Art. 10 (model card, data summary, limitations), and record in writing exactly what is unavailable. A liability gap acknowledged on paper is better than a gap discovered by an auditor.

Provocation

The three requirements of this chapter share one property: they physically cannot be produced after the fact. A risk management system assembled for the audit date proves exactly the opposite of what the law demands — that during design no one managed risks. Data governance presented after the fact honestly answers the auditor "we don't know what this was trained on." In high-risk compliance, the winner is not the one who writes documents fastest, but the one who started keeping them earliest.

Read next

• Next: Obligations II — Human oversight, logs and post-release monitoring
• All course chapters →
• Connect the expert as an MCP →