Models That Choose, Not Generate: Why the Decision Layer Is Becoming Its Own Category

TypeSafe AI released Jev on 15 September, calling it a System One Model.

Within a fortnight, open alternatives had appeared. Laya offers an open System One-style model. Researchers at Stanford and Nvidia released CLM-8B, which attacks the same class of bounded decisions with a contrastive architecture instead.

I would not yet call System One an established model category. Three releases in three weeks is a signal, not a settlement.

But something real has been exposed, and it matches what I have been building against for months: a large share of what we ask language models to do is not generation at all. It is selection.

The work we have been mislabelling

Most production AI systems make decisions that sit in an awkward middle.

They are too fuzzy for hand-written rules. They do not need a generative model either.

• Choosing which tool an agent should call
• Routing a request to the right specialist
• Deciding whether a guardrail has been tripped
• Picking the best of several candidate answers
• Scoring urgency, sentiment or risk
• Ranking retrieved documents before they enter a context window

None of these require the model to write anything. Every one of them has a known, finite answer space. And yet, in most systems I have seen, they are handled by asking a frontier model to produce prose and then parsing that prose back into a value.

That is the part worth naming. We take a classification problem, hand it to an autoregressive model, pay for token-by-token decoding, and then write a parser to undo the generation we did not want in the first place.

The parser is the tell. If your code has a regular expression whose job is to recover a route name from a sentence, you are paying a generation tax on a decision.

What actually changes at the interface

Large language models are autoregressive. They produce output one token at a time. That is exactly right when the output is text or code, where the shape of the answer is not known in advance.

It is unnecessary when you only want to know whether a support ticket belongs to billing, sales or technical.

A decision model inverts the interface. You give it the application state, the questions and the permitted answers. It returns typed values with probability and confidence attached. Instead of generating "This looks like a billing issue" and parsing it, the application receives billing.

Because the answers are scored rather than decoded, they come back in parallel rather than one token after another. TypeSafe reports 70 to 500 milliseconds end to end for Jev, $0.042 per million input tokens, and no metered output-token cost. For what they call System One shaped queries at comparable intelligence, they claim 40 to 200 times lower latency than frontier models and a 444.6 times cost reduction.

I treat vendor multiples with the caution they deserve — the comparison class is chosen by the vendor. The direction, though, is not controversial. Not decoding is faster than decoding.

Calibration is the underrated part

The latency numbers get the attention. I think the more consequential claim is calibration.

Jev is trained with Reinforcement Learning for Calibrated Decisions. The practical benefit is uncertainty you can actually act on: a spam score of 0.52 should not be treated like one of 0.998.

That distinction is what makes a decision layer operationally useful rather than merely fast. A calibrated system can automate the confident cases and escalate the ambiguous ones to a stronger model or a human. An uncalibrated one reports a coin flip as a certainty, and you find out in production.

This matters more than it sounds. Most of the value in routing comes not from being right on the easy cases — almost anything is right on the easy cases — but from knowing which cases you are not sure about.

Where the hallucination argument holds, and where it does not

A typed decision model cannot invent an output type. Its answers are constrained to the schema you define. In that narrow sense it removes a class of hallucination from decision tasks, which is a genuine improvement.

It can still choose the wrong answer.

And its ceiling is set by the options you provide. If a ticket is about fraud and the available answers are billing, sales and technical, the model cannot return the right category. It will return a wrong one, possibly with high confidence.

So the answer space becomes part of the system's quality. Designing those options — complete, distinct, mutually exclusive where it matters — is now engineering work, not configuration. I would treat a label set the way I treat a database schema: reviewed, versioned, and changed deliberately.

Where I would use one, and where I would not

Start with ordinary code. If the logic can be written as an explicit rule, write the rule. There is nothing to gain from putting probabilistic AI behind if balance < 0: require_payment(), and plenty to lose.

The case for a decision model appears when you know the possible answers but cannot easily write the rule that separates them.

That is a narrower band than the current enthusiasm suggests, and it is exactly where a great deal of production traffic sits.

One application I find genuinely interesting is replacing LLM-as-a-judge with a decision model as judge. Many systems already generate several candidates and use a second model to verify or rank them. A strong reasoning model can do the generating; a decision model can do the ranking; high-confidence outcomes get accepted immediately, and only ambiguous ones fall back to an expensive judge or a human.

That division of labour is the point. Expensive models do what they are uniquely good at — generation and reasoning. Cheap, bounded models do selection, routing and verification.

The open alternatives arrived immediately

Laya is an open-source alternative at 421 million parameters, built on ModernBERT with a dedicated decision head. It takes state and typed questions and scores every option in a single forward pass, and ships a Jev-compatible server API. Its developers report around 33 milliseconds for single-question inference on the multilingual checkpoint on a Tesla T4 — not directly comparable to a hosted end-to-end figure, but the right order of magnitude.

Contrastive Language Models take a different route. The 8-billion-parameter CLM maps the current state and the candidate actions into a shared embedding space and selects the best match. In the researchers' zero-shot tests it ran up to 9 times faster than Jev, with the largest gains where actions could be reused across many states or the candidate set was large — which is precisely the shape of a router with a fixed model pool.

The CLM team also tested the judge pattern directly. Larger models generated candidate coding solutions and a fine-tuned CLM ranked them: 81.6% on 38 DeepSWE tasks and 87.6% on 30 Terminal-Bench 2.1 tasks, against 71.1% and 83.1% for Jev. Small held-out sets, so I would not over-read them, but fine-tuning on the task distribution clearly moves the number.

That last point generalises. Every honest evaluation of this class of model I have read says the same thing: zero-shot performance is modest, and task-specific adaptation is where the quality comes from. A decision layer is not something you drop in. It is something you fit to your traffic.

What I have measured myself

I have been building on this directly rather than reading about it.

I published a research paper on Laya-MLX as a sub-15 ms decision layer last week, proposing three primitives — a guardrail gate, a complexity escalation ladder and a domain router — composed into a single batched call in front of a pool of specialised small models.

Running a three-question decision on an M1 Pro, our demo sits around 82 milliseconds median, on a 421M model, with nothing leaving the machine. That is slower than the 13.42 ms single-question figure published for an M3 Max, and it should be: three questions cost three encoder rows, and an M1 Pro is not an M3 Max. Both are fast enough for a component that runs before every request, which is the only threshold that matters.

The number I care about more is the second one: zero bytes to the cloud. A decision layer small enough to run locally changes where the decision can be made, not just how quickly.

The stack is becoming heterogeneous

Autoregressive models remain essential wherever the work is genuinely open-ended — hard reasoning, new text, new code.

But routing every intelligent operation through one general-purpose model is a habit, not an architecture. It adds latency and cost to operations that never needed a generative model at all.

What I see forming is a stack with three tiers rather than one:

• Conventional code for anything that can be stated as a rule
• Decision models for bounded choices that resist rules
• Generative models for open-ended work

This is the same argument I made about model orchestration as the decisive new SDLC, now with a concrete new primitive to build it from. The orchestration layer needed something cheap and calibrated to sit at its front door. It is starting to get one.

Jev, Laya and CLM will not all survive in their current form. The category may not even keep its name.

But the underlying observation will hold: most of what production AI systems do all day is choose, not write. Building infrastructure that reflects that is how expensive intelligence becomes affordable enough to run at scale.