AnyJev guide: turn an open LLM into a local Jev-style decision layer
Learn how Nokia Applied Research’s AnyJev adds Choice, Score, and yes/no decisions to open LLMs, how L0/L1/L2 differ, and when to choose it over hosted Jev, Laya, JevK5, or Winnow.
AnyJev is worth testing when you already run an open model such as Qwen and care about classification quality, calibrated probabilities, and local data. It is neither a new base model nor TypeSafe AI’s official Jev. It adds typed questions, debiasing, calibration, and an optional closed-form decision head to an existing LLM.
AnyJev is a decision readout for an existing LLM
AnyJev is published by the Nokia Applied Research organization, with authors affiliated with Nokia and Tencent Hunyuan. It reads an open LLM’s next-token distribution or an intermediate hidden state as a bounded decision: Choice selects a named option, Score places a case on an ordered scale, and yes/no estimates whether a proposition is true. It generates no prose and requires no JSON parsing.
That is why this directory classifies it as a compatibility layer rather than a separately trained System One model. It is not affiliated with TypeSafe AI, and a similar interface does not establish equivalent quality, latency, or probability semantics.
Local evidence
Qwen / other bases
Debias / calibrate / head
Choice / Score / yes-no
L0, L1, and L2 solve different problems
Do not reduce the three levels to “faster classification.” L0 addresses option-position and label-prior bias; L1 calibrates confidence with labeled cases from the same question; L2 solves a closed-form head tied to one question and one model.
| Level | Needs | Provides | Does not guarantee |
|---|---|---|---|
L0 | No labels | Option rotation and prior correction to reduce ordering bias | Calibrated probabilities; Choice costs K prefills |
L1 | 100–500 labels per question | Temperature scaling over L0 for better calibration | Changing a wrong ranking or surviving distribution shift |
L2 | Typically 100–300 labels per question plus hidden states | A closed-form head, often read partway through the model | Transfer to another question or model |
Start at L0, then decide whether labels are worth collecting
The package currently requires Python 3.10 or newer. This example loads Qwen locally through the Hugging Face backend and explicitly requests L0. Begin with a reversible routing task that is easy to inspect manually.
python -m pip install "anyjev[hf]"from anyjev import Decider, Question
from anyjev.backends.hf import HFBackend
decider = Decider(
HFBackend("Qwen/Qwen3-4B"),
level="L0",
)
route = Question.choice(
"Which team should handle this ticket?",
["billing", "technical", "sales", "human"],
name="route",
)
state = "Enterprise checkout fails after payment confirmation."
decision = decider.decide(state, [route])["route"]
print(decision.argmax)
print(decision.distribution)
print(decision.level)The goal is not immediate automation. Build your own comparison set: record the input, human label, AnyJev level, full distribution, latency, and final outcome. Once a stable label stream exists, calibrate L1 or fit an L2 head for a fixed, high-value question.
The published results are useful evidence, not your production result
On 300 BANKING77 20-way test items with Qwen3-8B, the repository reports accuracy moving from 0.747 raw to 0.803 at L0, while answer flips under reversed options fall from 0.230 to 0.073. With 100–500 labels, L1 reduces ECE from 0.240 raw to 0.095. Its L2 table uses 20 typed-decisions questions, with 300 labeled cases to fit each question and 100 held out for testing.
Five boundaries to check before production
- You still operate the base model
It reuses an existing LLM and does not inherit the memory, cost, or operational profile of a tiny dedicated classifier.
- L0 uses multiple prefills
A K-option Choice normally uses K rotations; yes/no uses two and Score uses one.
- L2 is tied to a question and model
After changing the question, option set, or base model, the old head cannot be assumed valid.
- Calibration cannot add missing knowledge
Rescaling confidence cannot make the base model solve a task it does not understand.
- The package is still pre-alpha
Pin dependencies, keep offline evaluations, default to review, and revalidate after every upgrade.
Choose around the assets you already have
Keep data local and collect labels for stable questions.
Accept a dedicated deployment or fine-tuning path to prioritize size and inference cost.
Inspect interface, calibration evidence, license, and maintenance one project at a time.
Avoid base-model and label-pipeline operations, while accepting a hosted API.
The safest evaluation runs the same real cases through each candidate and compares quality, calibration, abstention coverage, end-to-end latency, hardware cost, and maintenance burden together.