All posts · · 5 min read
A judge with no vendor
The calibrated probability that decides what a person sees no longer has to come from an API. An open-source System One checkpoint on your own disk answers the same typed questions with no key, no network and no cost per token — and the same program runs against either one.
The argument for a System One model is that a calibrated probability is the control you need to automate a decision: above the threshold the machine acts, below it a person looks. We made that case with Jev, the first model of the class, and the operational side of it in Putting a System One model in production.
Two objections came back, and they are the same objection twice. We cannot send those tickets to a third party. And: we are not putting a decision that routes our queue behind one vendor's API.
Since engine v0.6.27 the answer to both is a configuration change. The judge slot takes a third backend: an open-source checkpoint on a disk you control, answering the same three typed questions — is this true, which of these, where on this scale — with no key, no network and no cost per token. The program does not change. Not a line.
What the model is§
Laya, from Convai Innovations, Apache 2.0: a 421M-parameter ModernBERT backbone with a decision head that scores typed questions about a state in a single forward pass. It is the same class of model as Jev, published under a licence that lets you put the weights inside your own deployment, in your own region, inside your own enclave.
Concretely, the checkpoint is about 843 MB, it runs on CPU, and the engine ships with the ability to read it — so wiring it is two environment variables: which backend, and where the checkpoint is. Nothing is downloaded automatically; the weights are provisioned like any other artifact.
What it looks like in production§
The same block that asked four questions about a support ticket through the API answered against a local checkpoint on a laptop with no GPU: a refund request at 0.86, the right team (billing) but with the mass spread — 0.36 of confidence — so the gate sent the ticket to a person, and an off-topic message correctly matched none of the options at 0.86. Five questions in two blocks took 7.9 seconds of wall clock with the model load included; one question in a fresh process takes 2.5 seconds. Three runs gave identical numbers, to the fourth decimal.
Read those two sentences together, because that is the trade in one paragraph. You give up the vendor's 70–500 ms and you get seconds on a CPU. In exchange the state never leaves the machine, the per-token cost is zero, and there is no provider whose outage is your outage. For a queue that is triaged in batches, or an agent that decides once per document, seconds is invisible. For a synchronous path in front of a user typing, it is not — and there the hosted model is still the right call.
The same knob that routes decide through the judge works here too, which means a decision primitive that runs with no network capability granted at all: the process has the right to judge and no right to make a request. There is nothing to deny, because there is nothing to call.
Two things to do before you switch§
Re-measure your threshold. A 0.9 gate is a number tuned against a specific model. Jev and Laya are different models — the shape of the answer is identical, the calibration is not necessarily. Run a week of traffic through both, compare each answer against what the human eventually did, and set the number from that. This is the same discipline you would apply to any scoring model you changed, and it is the only honest way to move it.
Know the window. The local checkpoint reads 512 tokens: the question, the options and then as much of the state as fits. A long thread gets its tail cut, silently, because a truncated state still produces a perfectly plausible answer. Judge the fields that matter — the last customer message, the subject, the amount — rather than the whole object. That is better practice against the API too; it is mandatory here.
What you do not have to worry about is a fabricated number. When a judge is unavailable, over budget, or not wired at all, every answer comes back marked unavailable with confidence 0 and no value — so the gate you already wrote routes the queue to humans by itself. An invented sentence is visible in your output; an invented probability is not, and it multiplies into money.
Why this makes the slot worth building on§
A judge backend is a protocol, not a product. That was a design decision in the language before there was anywhere else to point it, and it is now a portability property you can check: the same program, the same block, the same gate, running against a hosted API in one deployment and a file on disk in the next. The customer who cannot send you their data and the customer who wants the lowest latency are served by the same code path.
It also answers the procurement question that has no good answer when there is one provider. What happens to this feature if that company changes its pricing, its terms, or its mind? You move a variable. The decisions keep being made, in your own datacenter, from weights whose licence you already have.
The developer-side guide, verb by verb with the numbers, is Run the judge locally with Laya. The generative half of the same release — a model in your process, with no egress — is Inference that never leaves your machine.