Every data pipeline starts the same way: a stream of raw records lands at the entrance, and the first thing you need is to figure out what each record actually is. What type, what format, what attributes, whether it's worth spending expensive downstream steps on. At this point, teams usually pick between two bad options: hand the work to a large language model \u2014 expensive and slow on every single record \u2014 or hardcode it into regexes and heuristics \u2014 cheap, but brittle and blind to meaning.<\/p>\n
A third option appeared recently: System One<\/strong> models. I spent time with one of them, Jev from TypeSafe, and the conclusion I want to lock in is this: the most valuable place for such a model is not somewhere in the middle of the pipeline. It is at the very entrance, as a labeling and enrichment layer.<\/p>\n <\/p>\n A large language model is "System Two" in Kahneman's terms: it reasons, generates text, spends time and tokens on thinking. System One is reflex. Jev does not write text at all. It takes a "state" (a chunk of data) and a set of typed questions<\/strong>, and returns typed answers:<\/p>\n All questions resolve in a single parallel pass over the same state. In practice: ~400 ms and fractions of a cent per record with a dozen or so questions \u2014 an order of magnitude or two cheaper and faster than running the same classification through an LLM.<\/p>\n The key property: the answer cannot<\/strong> exist outside the schema you define. The model physically cannot generate "and another thing I noticed\u2026". For labeling, this is not a limitation \u2014 it is the feature. You get exactly the fields you asked for, in a predictable shape. That property determines where the model belongs.<\/p>\n A raw record at the entrance is almost always under-labeled. It has a couple of obvious fields and a large chunk of unstructured text from which all meaning still needs to be extracted<\/strong>. And that meaning is needed immediately by everyone downstream: the filter gate (should this pass through?), the router (which branch?), the scorer (how to evaluate?).<\/p>\n The classic anti-pattern is extracting these attributes later<\/strong>, inside an expensive LLM step, and doing it from scratch every time. You pay for reasoning where you need classification, and the meaning gets locked inside a single step \u2014 inaccessible to everything upstream.<\/p>\n System One at the entrance inverts this. One cheap call, and the record is enriched: format, category, a set of boolean flags, a grade. From that point on, the entire pipeline reads structured facts from fields<\/strong> instead of re-reading the text. The gate sees what it could not see before. The scoring step gets an input it used to produce for itself. Policies (what to do with what) move into config and read ready-made attributes \u2014 without a single additional model call.<\/p>\n A cheap model at the entrance is not a solution by itself. Three techniques make it production-ready.<\/p>\n 1. Confidence is a routing signal, not decoration.<\/strong> Every answer arrives with 2. Calibrate the threshold \u2014 do not grab 0.5 from the ceiling.<\/strong> When I cross-checked System One answers against a gold-standard label set, nearly all disagreements clustered in a narrow band around 0.5 \u2014 the zone where the model is honestly uncertain. For boolean 3. Roll to production through shadow mode.<\/strong> A new classifier cannot become authoritative immediately \u2014 it can silently break decisions. The correct path: first it labels in parallel<\/strong> with the old mechanism, and disagreements are only logged. You collect telemetry, see where and how it diverges, calibrate \u2014 and only then flip the flag: "your answers are now primary, the old model is a validator on disagreements." One toggle in config, instant rollback.<\/p>\n The temptation is to route everything through the cheap model. But the boundary runs exactly where the boundary between classification and generation runs.<\/p>\n The right architecture is not "replace the LLM." It is strip the cheap labeling layer off it<\/strong> and leave the LLM exactly what you keep it for: reasoning and generation.<\/p>\n Threshold at 0.5, uncalibrated.<\/strong> The default cutoff sits squarely inside the drift zone. You ship a classifier that flips on borderline records between runs, and the pipeline treats identical inputs differently on Tuesday and Wednesday. Calibration against a labeled corpus is not optional \u2014 it is the difference between a classifier and a coin.<\/p>\n Making the new classifier authoritative on day one.<\/strong> A System One model that silently mislabels a category breaks every downstream decision that reads that field. Shadow mode first, telemetry second, flag flip third. Skipping the first two steps means you find out about the breakage from users, not from logs.<\/p>\n Routing nuanced judgment through System One.<\/strong> It returns typed answers against a fixed schema. If the question requires weighing context against a rubric \u2014 not extracting a fact \u2014 the cheap model gives you a confident answer to the wrong question. Measure the shift on a labeled corpus before moving anything nuanced.<\/p>\n Expecting free text.<\/strong> System One does not generate. If a downstream step needs a summary, an explanation, or a extracted quote, that step needs an LLM. Pushing that work onto a classifier produces an empty field and a broken pipeline.<\/p>\n System One at the pipeline entrance is the missing cheap layer between brittle regexes and expensive LLMs. It turns a raw stream into labeled records for fractions of a cent, returns confidence as a cascade signal, and \u2014 with careful calibration and a shadow rollout \u2014 takes over all routine classification without touching the nuanced core.<\/p>\n The value is the shift in order. Not "extract meaning when needed," but "label at the entrance, once, cheaply, and the entire pipeline reads prepared facts." Moving one step to the front changes the economics of the whole conveyor.<\/p>\n Count how many times your pipeline re-extracts the same attributes from the same text. Each extraction is a model call you already paid for once \u2014 and will pay for again on the next record.<\/p>\n","protected":false},"excerpt":{"rendered":" A cheap classification layer at the pipeline entrance changes the economics of the entire conveyor. One call turns raw records into labeled facts \u2014 every downstream step reads fields instead of re-extracting meaning.<\/p>\n","protected":false},"author":0,"featured_media":207,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[24,215],"tags":[217,219,218,220,216],"class_list":["post-208","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-engineering","category-data-pipelines","tag-classification","tag-data-enrichment","tag-llm-cost","tag-pipeline-architecture","tag-system-one"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\tWhat System One Is \u2014 and What It Is Not<\/h2>\n
\n
Why the Pipeline Entrance<\/h2>\n
Three Techniques That Make It Work<\/h2>\n
confidence<\/code>. Low confidence on a specific field is a free trigger to escalate that particular record to an expensive model, and accept the rest as-is. Cheap by default, expensive only on the contentious cases. System One and the LLM work as a cascade, not an either-or.<\/p>\nnoul<\/code> flags, the output drifts between runs<\/strong> in that zone. Raising the cutoff from 0.5 to ~0.65 eliminated all disagreements at once: the boundary landed above<\/strong> the drift zone, and the output became both more accurate and more stable. Confident answers (extreme values) were untouched. The rule: look at the distribution of noul scores, place the threshold in the gap between "not sure" and "confidently yes."<\/p>\nWhere System One Does Not Help \u2014 Honestly<\/h2>\n
\n
Failure Modes<\/h2>\n
Label Once at the Entrance<\/h2>\n