LLMs vs Jev vs CDMs: three ways a model can answer

Most of the model calls I write are not conversations. They are narrow questions with a short answer: is this PDF an invoice, which team owns this ticket, what is the total on page two. I still send them to a language model, and the model still writes its answer out one token at a time, even when the answer is a single word.

In the last six weeks two products launched that skip the writing. TypeSafe released Jev on 15 September 2026, a "decision model" that returns probabilities instead of text. ScaleDown followed on 13 October 2026 with Constrained Decoding Models (CDMs): Classify, Extract and Summarize, which it says read the input once and return structured output. Both come with big speed and cost multiples, and both have already drawn independent tests and critiques.

I read the docs, the launch posts, the two arXiv papers and the skeptical write-ups, because the question for me is practical: for the PDF extraction, classification and grounded content features I build, which of these three should answer a given call? This post is what I found, with the vendor numbers marked as vendor numbers.

The short version: an LLM generates, Jev chooses, and a CDM picks values out of what you gave it. Everything else follows from that.

LLMs: the loop that writes

An autoregressive LLM produces output with a loop. Each forward pass reads the whole sequence so far and computes a probability distribution over its vocabulary for the next token. One token is sampled, appended to the sequence, and the model runs again. Ten tokens of output means ten passes. A JSON object with six fields is dozens of passes, and a reasoning model adds its hidden thinking tokens on top.

Structured outputs do not change that loop. When I pass a JSON schema, the API constrains which tokens are allowed at each step so the result always parses, but the model still writes every brace and quote. That is why a "constrained" answer from an LLM is reliable in shape and still costs output tokens and time.

Cost follows the same shape. You pay for input and output tokens, and output is priced higher. On Anthropic's current list, Claude Haiku 5.5 is $0.10 per million input tokens and $0.50 per million output tokens (for prompts up to 100K tokens), and Claude Sonnet 5.5 is $2 and $10. Latency is time to first token plus the number of output tokens times the time per token.

Good at, bad at

• Good at: anything open-ended. Free text, explanations, drafting grounded content with citations, multi-step reasoning, tool use, and messy inputs where you cannot list the possible answers in advance.
• Good at: extraction when the fields need judgment, such as "the total after the credit note" rather than "the number next to Total".
• Bad at: being cheap and fast on a high-volume, narrow question. You pay for a writer when you needed a switch.
• Bad at: honest confidence. You get an answer, not a calibrated probability for each option, unless you build that yourself.

When I reach for one: whenever the output is language, or the task needs reasoning I cannot reduce to a fixed set of answers. For PDF extraction I still use an LLM with a schema, and I ask for the quote each value came from so I can check it against the document.

import Anthropic from "@anthropic-ai/sdk";
import { z } from "zod";
import { zodOutputFormat } from "@anthropic-ai/sdk/helpers/zod";
import fs from "fs";

const Invoice = z.object({
  vendor: z.string(),
  total: z.string(),
  invoice_date: z.string(),
  quote_for_total: z.string(), // exact text the total came from
});

const client = new Anthropic();
const pdf = fs.readFileSync("./invoice.pdf").toString("base64");

const response = await client.messages.parse({
  model: "claude-haiku-5-5",
  max_tokens: 1024,
  output_config: { effort: "low", format: zodOutputFormat(Invoice) },
  messages: [
    {
      role: "user",
      content: [
        {
          type: "document",
          source: { type: "base64", media_type: "application/pdf", data: pdf },
        },
        { type: "text", text: "Extract the vendor, total and invoice date." },
      ],
    },
  ],
});

if (response.stop_reason === "refusal" || !response.parsed_output) {
  throw new Error("No usable extraction; send to review");
}
console.log(response.parsed_output);

parsed_output is null if parsing failed, so the code treats that, and a refusal, as "send to a human". Low effort keeps Haiku fast on a task that needs little thinking.

Jev: a decision model from TypeSafe

Jev is made by TypeSafe AI, a startup that calls it the first of its "System One" models. Early access opened on 15 September 2026, and Jev 1.14 shipped on 14 October. You send it a state (text, and since 1.14 images) and one or more typed questions. It answers four kinds: choice picks one option from a set, noul gives the probability that a yes/no statement holds, score places the input on an ordered scale, and box finds a target in an image. It never returns prose or a reasoning trace.

How it computes that answer is mostly not public. TypeSafe's launch post says Jev uses a new architecture, a parallel sampler that produces all outputs from one query, and a training method it calls Reinforcement Learning for Calibrated Decisions (RLCD). There is no paper, no published weights and no reward function. So the honest description is "an interface we can see, an engine we cannot".

Is Jev related to JEPA? I checked because the names look alike and one Medium post says Jev follows the Joint Embedding Predictive Architecture idea. TypeSafe's launch post, its docs and OpenRouter's Jev guide do not mention JEPA at all. I found no primary source that links them, so I treat the claim as unverified. The only thing they share that I can confirm is the letter J.

The response is easy to branch on. In OpenRouter's tutorial example the ticket gets noul: 0.96 for "is this a bug", choice: "payments" with probability 0.78 and confidence 0.67, and an urgency score of 1.99 on a 0 to 2 scale. Every question in a request sees the same state and none can see the others' answers.

Cost and latency

Jev costs $0.042 per million input tokens and output is free, both on TypeSafe's own API and through OpenRouter. TypeSafe reports 70 to 500 ms end to end against 3 to 329 seconds for frontier LLMs, and headlines "193.6x faster, 444.6x cheaper" from its own workflow evals. The launch post itself says those numbers are likely the high end, that the evals were built by its own team, and that it cannot rule out subsidized pricing. Context is about 32K tokens for state plus one question, with a long-context mode up to 262,144 tokens that TypeSafe describes as slower and less accurate.

Where it fails

Typed output means Jev cannot return an answer outside your schema, but it can return the wrong valid answer. A September paper by Sun and colleagues tested Jev and two open-weight decision models by changing only which option name was bound to which definition. With "yes" and "no" as names, decision flips were up to 70.4 points higher than with neutral 0/1 names, and reassignment pushed mean AUC from 93.8% to 23.2%, below chance. The type-error rate stayed at 0% the whole time. My takeaway: name your options neutrally or descriptively, and put the meaning in the criteria.

On the positive side, Delip Rao and Chris Callison-Burch compared Jev and Perplexity's open reproduction, Decider, with three flash-tier LLMs as rubric judges across nine panels. Each decision model trailed the LLM judges significantly in at most 6 of 27 comparisons, while the LLM judges cost 16 to 357 times as much and took 5 to 350 times as long. On the independent DecisionBench leaderboard, Jev 1.14 sat 4th of 18 decision models when I checked, with an overall score of 69.

Jev does have a public API. The simplest route is OpenRouter's Decisions API, which needs only an OpenRouter key:

// From OpenRouter's Jev tutorial (one of its three questions shown)
const response = await fetch('https://openrouter.ai/api/alpha/decisions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'typesafe/jev-1.13',
    state: {
      customer_tier: 'enterprise',
      ticket:
        'My checkout page shows a blank screen after I click Pay. I have tried two browsers.',
    },
    questions: {
      team: {
        type: 'choice',
        instructions: 'Which team should own this ticket?',
        criteria: {
          payments: 'Checkout, billing, or payment processing issues.',
          frontend: 'Rendering, layout, or browser compatibility issues.',
          account: 'Login, permissions, or profile issues.',
        },
      },
    },
  }),
});
const result = await response.json();
console.log(result.answers);

When I would pick it: routing, tagging, triage, guardrail checks and LLM-as-judge rubrics, anywhere I currently prompt a model and parse one word out of its reply. I would pin the version, because answers shift a little between releases and between calls.

CDMs: ScaleDown's read-once models

ScaleDown launched three Constrained Decoding Models on 13 October 2026. Classify returns a probability for each label you define, each label backed by a yes/no rubric question, plus an optional one-sentence reason. Extract takes a JSON schema whose property descriptions say what to find, and returns each match with its text, a confidence, character offsets and surrounding context. Summarize returns a summary under 1,000 tokens from long text, images or PDFs. Compress, which trims context before an LLM call, is an older and separate ScaleDown model. Inputs can be up to 1M tokens, and pricing is $0.05 per million input tokens with free output.

ScaleDown's description is that each CDM reads the input once and skips the token-by-token loop. That fits Classify (a score per label) and Extract (spans with offsets). It does not fit Summarize, which the docs describe as abstractive, a fluent rewrite in the model's own words. That is generation.

The launch claims are 200x faster and 467x cheaper than frontier LLM APIs, and 77.2% on DecisionBench against 72% for Jev. These are ScaleDown's own figures, and no methodology for them was published with the launch.

What the critiques found

The sharpest critique, from ibl.ai, makes three points I agree with. Constrained decoding is an existing decoding-time technique, and tools like Outlines and XGrammar already force 100% schema validity on ordinary models without retraining, so "new architecture" is a product name. Summarize still generates. And valid output is not correct output.

The numbers also do not line up with ScaleDown's own docs. Its head-to-head benchmark page reports 2.2x to 13.9x faster against each provider's average, and against Anthropic's lineup 3.8x faster and 161x cheaper. I could not find what the 200x was measured against. For the DecisionBench figure, I found no public source for 77.2%, and the independent DecisionBench leaderboard I could find ranks ScaleDown Classify-1 last of 18 decision models, with an overall score of 6 on a scale where the weakest model on each test gets 0. It may not be the benchmark ScaleDown meant, but it is the one with that name and published data. I also noticed that ScaleDown's headline multiples sit very close to TypeSafe's own "193.6x faster, 444.6x cheaper". I cannot tell whether that is coincidence.

The most useful test I found was a hands-on review on DEV Community that ran the CDMs on public labelled datasets. Classify scored 96.7% on a 300-sample CLINC150 slice with a median latency of 134 ms. Extract matched a small local model on receipts (89.5% field accuracy each) but was faster, and every extracted span appeared verbatim in the source. Summarize was fast and invented no facts on six samples, but on very long transcripts it sometimes summarized the wrong section.

Cost is where the headline breaks down most. The reviewer found ScaleDown billed more input tokens than the raw prompt (about 481 against 172 for a classify call), so per 1,000 short classifications it cost $0.024, twice gpt-5-nano's $0.012 at list price. Against flagship models it was roughly 45x to 200x cheaper. The 467x only appears if the comparison model also burns hundreds of reasoning tokens per call.

The same review caught Extract picking the buyer instead of the seller on one receipt template. No schema validator flags that, because both names are strings in the right place. Offsets help here, since I can show the user exactly where a value came from, but only labelled examples tell me how often the value is wrong.

The API is documented and easy to try. Extract is OpenAI-compatible, so the docs use the OpenAI SDK with ScaleDown's base URL:

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.scaledown.xyz/v1",
  apiKey: "<your-api-key>",
});

const r = await client.chat.completions.create({
  model: "extract",
  messages: [{ role: "user", content: "Acme Corp invoiced $500 on 2024-01-05." }],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "invoice",
      schema: {
        type: "object",
        properties: {
          vendor: { type: "string", description: "company name" },
          amount: { type: "string", description: "dollar amount" },
          date: { type: "string", description: "invoice date" },
        },
      },
    },
  },
});
const result = JSON.parse(r.choices[0].message.content!);
console.log(result.entities);

Classify is a plain POST with your labels and rubrics:

const response = await fetch("https://api.scaledown.xyz/classify", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "x-api-key": "<your-api-key>",
  },
  body: JSON.stringify({
    text: "The patient presents with a high fever and difficulty breathing.",
    labels: [
      {
        name: "medical",
        rubric: "Does this text describe a medical condition, symptom, or health topic?",
      },
      {
        name: "legal",
        rubric: "Does this text describe a legal matter, contract, or regulatory issue?",
      },
      {
        name: "financial",
        rubric: "Does this text describe a financial transaction, investment, or monetary matter?",
      },
    ],
  }),
});
const data = await response.json();
// data.top_label === "medical", data.scores.medical === 0.887 in the docs' example

ScaleDown also ships a /v1/scaledown endpoint that accepts Jev's request shape, so a Jev integration can be pointed at it by changing the base URL. That makes a side-by-side test on your own data cheap to set up.

Same answers, same mistakes

The finding I keep coming back to is in the Rao and Callison-Burch paper. On a decision model's most confident errors, over 85% of flash-tier LLM verdicts repeated the same error, against about half if the errors were independent. Cascades that sent uncertain cases to an LLM gained at most 2.7 points over the best single judge. So the common plan of "cheap model first, LLM on low confidence" mostly saves money. It does not catch the mistakes the cheap model is sure about. Only labelled data does that.

Which one I would pick

This is the process I am using for the narrow calls in my own features:

1. Write down the output type

   Free text, one of N labels, a probability, a score on a scale, or a value copied from the input. If you cannot write it as a type, it belongs to an LLM.

2. Label 200 real examples

   Pull them from your own traffic, not a public dataset. Include the ugly ones. This is the only number that will matter later.

3. Run all three on the same set

   An LLM with structured outputs at low effort, Jev, and the matching CDM. ScaleDown's Jev-shaped endpoint makes the last two nearly the same code.

4. Measure what you pay for

   Accuracy, p50 and p95 latency, and cost per 1,000 calls from the billed usage, not from list prices. Check billed tokens against your prompt size.

5. Set thresholds from the cost of mistakes

   Pick confidence cut-offs per label from your labelled set. Remember that a confident wrong answer will often fool the fallback too.

6. Pin versions and re-run on upgrades

   Use typesafe/jev-1.14, classify-1 or extract-1 rather than the latest aliases, and repeat the eval before you move.

My opinion after all this reading: decision models are a real and useful category, and for routing and tagging I expect to move calls off LLMs. The marketing around them is ahead of the evidence. "Cannot hallucinate" means "cannot return an off-schema answer", and 200x or 467x is a ratio against a chosen baseline, not a property of the model. Treat the multiples as a reason to run your own test, and let your labelled examples make the call.

Sources