Track: Model Classes · Module 2 · Prereq: Module 1 (: models that decide instead of write) Hook: A decision model sells you a number. If you build one out of an ordinary LLM, you have to choose which number to read, and the server will happily hand you the wrong one without an error.
The 2-minute version (no math)
Module 1 described a new kind of model that answers one narrow question with a probability attached. You can get the same behavior from an ordinary open LLM: ask it a yes/no or pick-one question, and instead of reading the words it writes, read how much probability it puts on each possible answer.
The catch is that a language model's server does not compute just one set of probabilities. Before it picks each word, it adjusts the model's scores using whatever sampling settings you sent: , how many candidates to keep, penalties for repetition. Those settings exist to make generated text read well. They have nothing to do with what the model believes.
So there are two places to read the probability. Raw is the model's own view, before any of those adjustments. Processed is what is left after them. For a decision you should always read raw, because a processed number changes when someone edits a server config, and it can report a perfectly good answer as impossible.
The two labs below let you move the knobs and watch it happen.
Two places to read a number
At every step of generation, a model produces a score, called a , for every token in its vocabulary. A is just the logarithm of those scores after they are turned into probabilities, so the only question is which scores you take it from. Before vLLM picks a token, it passes the scores through a chain of adjustments:
Raw logprobs describe what the model believes. Processed logprobs describe what the sampler actually drew from. vLLM lets you choose with its logprobs_mode setting, which takes four values: raw_logprobs, raw_logits, processed_logprobs and processed_logits. The default is raw_logprobs. In vLLM's own definition, raw means the values before any logit processor runs, and processed means the values after all of them, "including temperature and top_k/top_p." Option names can change between releases, so confirm against the documentation for the version you run.
Two ways to build a decision model
Module 1 introduced Jev, the closed decision model TypeSafe announced on September 15, 2026, trained with a method TypeSafe calls Reinforcement Learning for Decisions (RLCD). Jev is API-only, so you only ever see its final probabilities. Within weeks, open-weight alternatives appeared, and they come in two families. The raw-versus-processed question matters very differently for each.
Family A: encoders with a decision head
Examples:
convaiinnovations/laya, built on ModernBERT-large.muthuishere/openjevx, derived from Laya: about 0.4B parameters, serving a Jev-shapedPOST /v1/systemoneendpoint.com-kotobalabs/open-jev-deberta-v3-large, built on DeBERTa-v3-large.
These read the whole input in one pass; a head then scores each allowed answer, everything outside the answer set is masked, and a softmax turns the scores into probabilities. Their SDKs expose typed questions such as a choice among options, a score, and a yes/no.
For these models, vLLM's logprobs settings do not apply at all. There is no sampling step, so there is nothing for "raw" or "processed" to be relative to: the probability is the output. If you want to adjust the confidence, that is a calibration job you do afterwards, not a sampler setting.
Family B: language models adapted to answer the same way
Examples:
openjev/openjev, a Qwen3.5 decoder despite sharing a name with the DeBERTa model above, served with vLLM and documented with--max-logprobs 64.bespokelabs/Bespoke-Nimble-9B, a LoRA adapter onQwen/Qwen3.5-9B.
These add "state in, typed probabilities out" to an existing open LLM. When the typed probability is built from token logprobs, the logprobs mode directly decides the number your code acts on. Check each project's code: some read logits internally rather than through vLLM's logprob API.
The recipe mirrors the encoder's head, built from vocabulary scores instead: take the logprob of each answer's token, mask out everything else, and renormalize. Lab 2 shows why doing that from processed logprobs goes wrong.
What the lab makes visible
can silently delete an answer. With top_k set and processed logprobs, a valid option that ranked below some filler token is set to −∞ and reads as exactly 0%. Exactly 0% is a claim that the option is never right; if it is ever right, that probability was wrong. You have lost both an answer and your calibration, with no error.
Sampling temperature is not calibration. A low temperature makes the top option look far more certain. That certainty came from the server config, not the model. If your model's raw probabilities are over- or under-confident, fit a correction on a labeled set and apply it yourself (temperature scaling is the classic one). It is the same arithmetic as the sampler's temperature, used for a different purpose.
Coverage is a free health check. Before renormalizing, add up the raw probability on all the valid answer tokens. If the model put most of its probability on "The" or a line break, the renormalized answer is built on a sliver, and the prompt or adapter needs work. Encoder decision models hide this, because the mask is applied before you ever see a number.
Side by side
| Jev (closed) | Family A: encoder heads | Family B: LLM served with vLLM | |
|---|---|---|---|
| Where the probability comes from | A trained decision output; internals not visible | Softmax over candidate scores, masked to the answer set | Token logprobs, masked to answer tokens and renormalized by you or the adapter |
| Do sampling settings matter? | None exposed | No: there is no sampler | Yes, if you read processed_*. No, with the default raw_logprobs |
| Calibration | Trained for it (RLCD); verify on your data | Depends on training; measure it yourself | Chat-tuned LLMs are often over-confident; measure and recalibrate |
| What you can inspect | Final probabilities and confidence | Head scores and weights | The full vocabulary distribution, including probability on invalid tokens |
| Typical cost | Hosted API | Very cheap: a sub-1B encoder answers in tens of milliseconds | Heavier: a multi-billion-parameter decoder, but one forward pass if you read only the first answer token |
What we checked
Two of this module's claims carry the rest, so we verified them directly rather than from documentation alone.
- The default is raw. vLLM's configuration source defines the four
logprobs_modevalues above and setsraw_logprobsas the default. - Raw is computed before sampling settings apply. That follows from vLLM's definition above, and a live sanity check agreed with it: on a vLLM 0.30.0 endpoint we asked one yes/no question repeatedly at temperature 0 and at temperature 1. Repeats varied slightly at both temperatures, by the same amount at each, so changing the temperature added no difference of its own. A small check like this cannot prove the property; the code defines it, and the check found nothing to contradict it.
Rules of thumb for using vLLM as a decision engine
- Keep
logprobs_mode=raw_logprobsfor any probability your code acts on. Use the processed modes only when you need the sampler's exact policy, such as importance ratios in reinforcement-learning training. - Make sure every option is actually returned. A
logprobs=Nrequest returns only the top N tokens (plus the sampled one), capped by the server's maximum, which defaults to 20 in vLLM. An option outside the top N is simply missing. Safer: score each option explicitly withprompt_logprobsby appending it to the prompt (for prompt tokens, raw and processed are identical), or choose option labels that are single, distinct tokens. - Decide how to handle multi-token options. Summing logprobs penalizes longer labels; averaging hides the difference. Whichever you choose, use it consistently and test it.
- Check what the model is reading. A decision head can latch onto an option's name rather than the definition you attach to it. Test it: swap which definition goes with which name and see whether the answer follows the definition. Name options so the name itself carries the meaning.
- Calibrate on your own labeled data. For every source, Jev included, plot predicted confidence against accuracy before you set a threshold to act on automatically, and leave margin around it.
- Log coverage. Track the raw probability on valid options for every request. A sudden drop is an early sign of prompt drift or a broken adapter.
The honest close (the Verdict)
A decision model's promise is that the number it hands you means something. With an encoder, the architecture guarantees that the number is a distribution over your options. With an LLM behind vLLM, you have to guarantee it yourself, and the first step is reading raw logprobs, so that nobody's sampling settings can leak into the answer.
None of this makes the number trustworthy on its own. Raw logprobs tell you what the model believes; whether that belief is well calibrated on your traffic is still a measurement you have to make.
Glossary delta
Logit · Logprob · Temperature (sampling) · Truncation (top_k / top_p)