Putting a System One model in production

A System One model — TypeSafe's Jev is the first — answers typed questions about a state with calibrated probabilities instead of prose. For the half of an agent that decides rather than writes, that removes the parser, the retry loop and the invented confidence in one move.

It also changes the operational conversation, and that is the part a platform has to answer. Here is what putting one into production actually involves.

Two slots, not a swap§

The judge does not replace the language model. It sits next to it: the judge decides, the LLM writes, and the ordinary setup has both wired. On the platform they are two independent configurations (SYNSEMA_JUDGE_ alongside SYNSEMA_LLM_), two sets of credentials, and two budgets.

That separation is also a permission boundary, and it is the first thing a security review will notice:

require judge      -- may classify
require llm        -- may generate

Neither grants the other. A service deployed with judge and without llm is a service that can measure and cannot write — it cannot be talked into emitting free text, and it cannot exfiltrate through it. For the routing, triage and eligibility parts of a product, that is a much smaller thing to defend.

The key never reaches the program§

TYPESAFE_API_KEY goes in as a sealed secret, like any other: stored on the control plane, injected when the container starts, shown in the dashboard as a name and a fingerprint. The program cannot print it, log it, concatenate it or return it from a route, and the host the call goes to is fixed by the runtime — a deployed .syn cannot redirect its judgments to somewhere else.

Every capability check the runtime performs lands in the audit trail, so "did this service classify anything last night, and how much did it spend doing it" is a query rather than an investigation.

A preflight you can put in the deploy§

The most boring failure in production is a service that starts fine and silently stops judging because a key never made it into the environment. The CLI answers that before the process starts, with no network:

$ synsema judge status
Key         TYPESAFE_API_KEY             ✗ MISSING
Model       jev-latest                   (SYNSEMA_JUDGE_MODEL, default)
Base URL    https://api.typesafe.ai      (SYNSEMA_JUDGE_BASE_URL, default)
Budget      (no ceiling)                 (SYNSEMA_JUDGE_BUDGET, default)
decide      LLM (default)                (SYNSEMA_JUDGE_DECIDE)

Exit 0 when it is live, 1 when it is not, so synsema judge status && synsema serve app.syn is a gate and not a ritual. The key is reported by presence; the value never appears.

The day the provider is down§

This is the question worth settling before the incident, because the alternative to a good answer is a system that keeps answering confidently with numbers it made up.

Without a key, over the budget, or after a network failure, every answer comes back available: false, with confidence: 0 and its main value nothing. One notice goes to stderr and the program keeps running. An invented probability is never returned — which matters more than it sounds, because an invented sentence is visible in your output and an invented probability is not: it just multiplies quietly into a refund, a route, a payout.

The useful consequence is that degradation lands in the branch you already wrote. A confidence gate at 0.9 is not met by a confidence of 0, so every ticket goes to the human path automatically:

when v.team.available and confidence of v.team >= 0.9
    route(v.team.choice)
otherwise
    approve "Route to " + text(v.team.choice) + "?"

On the platform that approve is a real queue: the approval shows up in the dashboard and in the API, notified by signed webhook, answered with a one-time token. An outage at the model provider turns into a busier inbox for a few hours, not a wrong decision at scale.

A budget is a knob, not a spreadsheet§

SYNSEMA_JUDGE_BUDGET is a hard ceiling on input tokens per process. At the ceiling, answers degrade to available: false without touching the network — the spend stops at a number you chose rather than at the end of the month. Set it per service, the same way you set memory.

Pin the version your thresholds were tuned against§

judge_model() returns the versioned id that answered the last call (jev-1.13.0), never the alias, and that is deliberate: vendors move latest. A threshold at 0.9 is a number tuned against a specific model, so pin SYNSEMA_JUDGE_MODEL to a version in production, and re-measure your gates when you move it — the same discipline you would apply to a scoring model you trained yourself.

Two more things worth knowing before the first rollout: identical requests move by a few hundredths (0.72 → 0.69), so tests assert winners and ranges and never equality; and SYNSEMA_JUDGE_PROVIDER=mock answers deterministically with no network and no account, which is how this belongs in CI.

What it replaces§

For classification-shaped work, it replaces a chat call plus a parser plus a retry plus a confidence you invented. For the rest — the reply a customer reads, the summary, the explanation — the language model is still the right tool, and both are wired at once.

The developer-side guide is How to use Jev from Synsema; the manual page is Judge. The next post is about the part your finance team will ask about: what this actually costs compared with what you are doing today.