# Jev as a decision layer > TypeSafe's Jev answers typed questions with probabilities instead of writing text. Where it earned a place in my agent setup, what the evaluations showed, and the gateway route that never worked. - Canonical page: https://rob-knight.com/articles/jev-as-a-decision-layer/ - Kind: Article - Published: 2026-10-10 - Updated: 2026-10-19 - Topics: agents, cloudflare Jev is a model from TypeSafe that doesn't write anything. You send it some state and a list of typed questions, and each answer comes back as a probability: a distribution over choices you supplied, a yes/no probability, or a position on an ordered scale. I've used it three ways in the last month. It does triage in my agentic journal, it reranked memory recall in an agent-memory service I ran for a day, and it powers a code-search tool that every agent home has installed. The journal is where it earned its place. On deciding which audience a conversation belongs to, it went from 72% accurate to 99%, and almost all of that gain came from what I put in the state, not from the model.
A machine read left to right. In: state, plain text, built up in three layers, each with the audience accuracy the same model reached on it: summary text only, 72%; plus project, directory and agent home, 87%; plus a roster of which projects go where, 99%. A note reads same model in every row. The state goes into a petrol block, Jev: typed questions in, one batched request, about 0.4 seconds a call. Out: probabilities, no prose, for four questions: audience, a choice; noise, yes/no; publish, yes/no; value, a score from 1 to 5. Confidence of 0.9 or more leads to an OK chip, one-click accept; anything lower leads to a WARN chip, a suggestion shown with its percentage. Along the bottom: code handles what it can, Jev makes the closed decisions, and a generative model writes all the prose.
The accuracy figures come from 200 conversations weighted back to 765, and the 0.9 threshold was set on a calibration half and checked on a held-out half.
## What Jev is TypeSafe calls Jev the first [System One model](https://docs.typesafe.ai/concepts/system-one). A request carries `state`, which is plain text, and `questions`. There are three question types: a choice over options you list, a Noul (a yes/no probability), and a score on a scale you define. Several questions can share one request. TypeSafe's documentation says these models return typed decisions and probabilities, not generated replies or explanations. In this interface Jev selects or scores the supplied answers; it does not return a generated title, summary or extracted free-text value. What it can do is make a closed decision quickly and cheaply. The rate I used for every cost below is USD 0.042 per million input tokens, with output free. All of those costs are estimates from token counts at that rate, not billed readbacks. Every call I make goes through my Cloudflare AI Gateway, which is where the first problem was. ## The route that never worked The first version of the journal had a Jev client before it had ever made a successful Jev call. It posted to the gateway's OpenAI-compatible route at `/compat/v1/systemone` with a model name. The gateway's `/compat/models` listing includes Jev with a price, so the app's price check passed. Then the call failed. A loop run on 25 September recorded one Jev start, an `invalid_response` and no completed comparison, and parked the work. When I later told an agent Jev had struggled "because of all the json output", I was wrong: no request had ever reached the model. On 27 September, while wiring up code search, an agent probed every route. The compat route returns HTTP 400 with code 2019, "Compatibility endpoint ... is not supported". The native `/typesafe/...` route returns 2008, "Invalid provider", even with a TypeSafe key stored on the gateway. The only route that answers is Workers AI, `workers-ai/run/typesafe/jev`. That route also has its own shape: no `model` field in the body, the answer wrapped in `{"state":"Completed","result":{...}}`, and 429s with no `retry-after` header. Each tool I pointed at Jev needed a small adapter for those three differences. Billing doesn't land where you'd expect either. On that route Jev is billed through Unified Billing via the account's auto-created gateway named `default`, and the TypeSafe key on my own gateway is never consulted. My gateway logs each Jev call at zero cost, so its spend limit never sees Jev. The `default` gateway is recreated with stock settings if it's deleted, so I copied my gateway's cache, DLP policy, spend limit and keys onto it. ## Triage in the journal The second version of the journal, which publishes [the agent log](https://convos.rob-knight.com/), writes a summary of every agent conversation with a generative model. Two decisions then gate what gets published: which of two audiences the conversation belongs to (the public log or a private one), and whether it's worth publishing at all. The generative model made a suggestion for audience, and it was often wrong in a systematic way. Repositories I work on for my employer live in my own GitHub organisation, and the model kept calling them personal. On 29 and 30 September an agent ran a triage evaluation. It took 200 of the 765 summaries in the catalogue: all 93 where the generative model's suggestion disagreed with the agent home the conversation ran in, plus 107 random agreements, with accuracy weighted back to the full 765. An independent Opus agent labelled them without seeing the existing suggestions. I then corrected the labels from my answers about which projects belong where, and ticked 46 publish calls myself. Jev only ever saw summary-derived state, never the transcript: project, working directory name, agent home, title, objective, narrative, outcomes and anything left unfinished. Each conversation got all four questions in one request. | Audience method | Accuracy | | --------------------------------------------- | --------- | | Agent home alone (sample, unweighted) | 61% | | Generative model's own suggestion | 80 to 83% | | Jev, summary text only | 72% | | Jev, plus project, directory and agent home | 87% | | Jev, plus a roster of which projects go where | 99% |
A horizontal bar chart of audience accuracy, one bar per method, on an axis from 0 to 100% with ticks at 0, 50, 80, 90 and 100. Agent home alone, 61%. Generative model's suggestion, 80 to 83%. Jev, summary only, 72%. Jev + project, directory, agent home, 87%. Jev + roster, 99%. The three Jev bars share one colour and a bracket spanning them reads same model, more state. Footnote: 200 conversations, weighted to 765; agent home row is the unweighted sample.
The two baselines bracket Jev's summary-only score, and the climb from 72% to 99% happens without changing the model.
The model was the same in every Jev row. The jump from 72% to 87% came from adding three fields that were already in the database, and 87% to 99% came from a roster I wrote in a few minutes. The remaining misses were projects missing from the roster, plus one summary that described an employer project as "personal EKS infrastructure", which no classifier reading that summary could have got right. The confidence figure is what made it usable. The app treats the gap between the two audience probabilities as its confidence. In the second round the 200 were split into a calibration half and a held-out half. On the held-out half, confidence of 0.9 or more covered 92% of conversations with no errors, and all five misses in the 200 scored below 0.2. So the inbox offers one-click accept only at 0.9 or above, and shows anything lower as a suggestion with its percentage. Publishing was harder. A plain "should this be published?" yes/no looked like the obvious gate, and it didn't transfer: a threshold set on the calibration half wrongly skipped 10 of the 27 held-out conversations I'd marked yes. A five-level `value` score did transfer, with an AUC of 0.89. Skipping at a value of 2.15 or below dropped 51 of the 100 held-out conversations and wrongly dropped 1 of the 27 yeses. A separate `noise` question at 0.8 or above flagged 21 conversations, none of which I'd marked yes. But the probabilities cluster in the middle and rarely pass 0.8, so only about eleven conversations scored high enough to recommend automatically. Value became a sort order and a "not worth it" filter. Auto-approve wasn't built. Every accept and override I make is logged, so a future decision on auto-approve has a record to argue from. Each call used about 1,650 input tokens, under USD 0.0001 per conversation, and took about 0.4 seconds. The evaluation's 1,200 calls came to about six cents. In production, triage runs after every new summary. A 14-day backfill triaged 1,050 of 1,050 conversations with no failures, and the first 1,110 triage rows cost about eight cents between them. The evaluation had two faults, both caught in the same session. The roster that went into Jev's question was also in the labeller's brief, which flatters the first round's figure of 95%. The second round measured against my roster directly and added the held-out split. The other was the cache. The gateway caches responses for 900 seconds, so the first "repeat run" that showed perfect stability was a free replay of cached answers. Rerun with `cf-aig-skip-cache: true`, 199 of 200 audience answers matched. ## A skip gate before summarising The same idea moved earlier in the pipeline. Before the journal summarises a session, regex rules discard the obvious noise. A Jev rule now sits beside them, asking a `throwaway` yes/no only about sessions the regex rules keep, and skipping at 0.5. In its evaluation it skipped 12 of 224 kept sessions, all correct skips, and none of the 202 yeses, for about eight cents across 1,882 calls. I rejected a "worth summarising?" question as the gate because it under-scored my agent-tooling work, which is most of what I publish. A second gate, `loop_prep`, catches loop preparation and admin sessions. Over three identical runs it caught 26 or 27 of 34 and skipped none of the five sessions that had to stay, but its false positives were 4, 4 and 5 out of 78 against a bound of 4, so it missed its own bound in two of the three. It's off by default and I turned it on anyway. On its first day it skipped 42 more sessions, none of them one I'd already given an audience or one that had to stay. ## Reranking agent memory Hindsight is an agent-memory service. I ran it for my agents for about a day in September. Recall pulls up to about 300 candidate facts and a reranker orders them against the query. I replaced Cohere's reranker with Jev, through a shim. The shim had the same three route differences as everything else, plus a fourth: Workers AI caps Jev's input at about 32,000 tokens, roughly 170 realistic facts. The first comparison failed on 74 of 76 queries with `max_tokens_exceeded`. The fix was a tournament. The shim ranks chunks of the candidate list, re-ranks the finalists, and returns synthetic scores that preserve the order, which is all Hindsight uses. One probe blamed a 403 on the `model` field, when the gateway was actually rejecting Python's default user agent. The comparison used 68 real recall queries taken from trace data plus 8 natural-language goals. Each reranker's top eight facts per query were anonymised and scored blind by an Opus judge. Jev scored 8.1 against Cohere's 6.2 and was better on 65 of the 75 judged queries. It was slower, with a median of about 2.0 seconds against 1.2, and Cohere's faster 0.28 seconds on a rerun was again the gateway cache. I first estimated the cost at USD 0.002 per recall, then corrected it to about 0.005, because large pools rank in several rounds. Jev went into production with Cohere as automatic failover. About a day later I decommissioned Hindsight entirely, because I wasn't getting enough value from agent memory to justify running it. So the reranking numbers are real but nothing uses them now. ## Code search jevgrep, `jg`, is an open-source tool that answers "how, why or where does this work" questions about a repository with ranked files and source excerpts, and its decisions come from Jev. Its provider endpoints were hardcoded, so pointing it at my gateway wasn't a configuration change. I forked it and added a Cloudflare provider that saves the gateway URL with a scoped token, drops the `model` field, unwraps the response and refuses redirects so the token can't follow one. I offered it upstream as [dzhng/jevgrep#4](https://github.com/dzhng/jevgrep/pull/4), which was closed unmerged on 17 October, so the fork stays. A refresh script merges my `fork-main` branch onto upstream on every update, rebuilds the binary, and vendors the skill into my plugin marketplace so every agent home picks it up. A wrapper refuses to search agent homes, `~/.ssh` or my credentials tree, because a search sends selected source to Jev. The fork also keeps provider traffic on HTTP/1.1 and runs a watchdog that stops a runaway process, after Node 26's HTTP/2 teardown once grew `jg` to about 200 GB. Verification ran 411 requests over three uncached runs with no failures, at a median of 101 ms per request. Workers AI rate-limits Jev on its own, though, and one search came back partial after 42 rate-limited responses. Upstream's 0.8.5, [released on 18 October](https://github.com/dzhng/jevgrep/commit/157715f88d3974bdbf0c91c5922b51a9847a1c3c), now treats resource limits as partial success. Use was thin at first. In the two weeks after rollout I can find about six real discovery searches across three repositories. The rest of the 26 `jg` calls in my history are setup and checks. Agents rarely chose it until I raised Claude Code's limits on skill description length, so the skill's "use me first" line actually reached them. Since 16 October the agent rules and the [loop contract](/reference/loop-contract/#3-dispatch) tell discovery to start with `jg`, so use is now mandated. I haven't found a case where `jg` found something grep wouldn't have, and I haven't reassessed whether it beats grep since the mandate, so I'm not claiming it does. ## Where it fits The first evaluation, on 17 September, had already said where Jev belongs. Against the journal's generative model it scored 3 of 8 on a task-type taxonomy where the model scored 5, and the report's verdict was not to replace the model. On a constructed four-way audience question it scored 8 of 8. On checking whether evidence supports a claim it scored 6 of 6, against the model's 5. It was about nine times faster and nine times cheaper. Rewording the task-type descriptions moved it from 3 of 8 to 4, and a two-stage hierarchy dropped it to 2. The categories overlap, so no rewording was going to fix it. So Jev sits between deterministic code and a generative model. Code handles what it can. Jev makes the closed decisions in one batched request. Anything below a confidence threshold goes to the generative model or to me, and the generative model writes all the prose. The journal runs that whole shape now. The reranker and code search only ever used the middle step.
A left-to-right pipeline. Deterministic rules, such as regex skips, come first. Next is the only filled stage, Jev: closed questions, one batched request, holding four question types: audience (choice), noise (yes/no), publish (yes/no) and value (score 1-5). A line labelled confidence >= 0.9? then branches. The yes branch ends at an OK chip, one-click accept. The no branch ends at a WARN chip, suggestion shown with its %, which leads on to generative model or me. Underneath, a separate lane fed from the deterministic rules: generative model writes the summary and all prose.
Only the middle stage is Jev; the journal is the one tool that runs the whole shape, while reranking and code search used the Jev stage alone.
The lessons that carried across: - Context did more than the model. Three database fields and a roster took audience accuracy from 72% to 99%, so check what a classifier was shown before blaming it. - Confidence decides where a call goes, not whether it's right. TypeSafe's [calibration explanation](https://docs.typesafe.ai/concepts/system-one) says calibration across groups does not guarantee an individual answer. In the first evaluation one wrong label had high confidence. Set the threshold on a calibration half and check it on a held-out half. - Batch the questions. Five in one request used 54% fewer input tokens and 81% less time than five requests. - Skip the gateway cache when evaluating, or a repeat run looks perfectly stable and perfectly fast. - A model in the gateway's model list, with a price, still wasn't callable on that route. ## Limits The sample sizes are small: 8 real cases in the first evaluation, 200 conversations in the triage one, and 75 judged queries for reranking. Labels came from an Opus agent corrected by me, and the reranking judge was also a model. Costs are token counts multiplied by the listed rate, and I haven't reconciled them against billing. Reranking ran in production for about a day. The figures are from `jev-1.13.0`, which is what `jev-latest` resolved to throughout.