Key findings from Realtime-Bench: the best model GPT-6-Astra scores 36%, while Jev answers in under 1 second as the fastest model, scoring 8%. More reasoning helps only when the clock is paused; with a real clock, slow thinking costs the win. Even with time frozen, many tasks remain unsolved.
alphaXiv says it trained a 4B model on 117k decision-making examples; it performs nearly identically to Jev except in sarcasm detection, and generalizes to unseen tasks like hate-speech severity.
Wasim Jalali shares Day 9 of Decision Bench: Clef Omni ranks last of 8 with 87.2% accuracy, compared to Jev 1.13 on hidden questions and noise handling. It is conservative and leaves most noise in.
Carlos Muñoz Ferrandis reports that in one week, using two A100s and only open datasets plus synthetic replies, their model beats Jev on 6 of 8 public guard benchmarks, achieves 0.936 mean AUROC vs Jev's 0.934, and runs at 90 ms per request on one A100.
The post cites a Tencent research paper evaluating Jeve's judgment under option order changes, showing Jev often fails even on solvable problems, aligning with the author's experience.
Ben Kolligs used Jev from @typesafe to classify titles and abstracts of robot modeling papers, finding VLA model papers rose from ~10% in 2023 to 38% in Jan–Sep 2026, and world models from 9% to 18%.
In the Latent Space podcast, swyx discusses Jev and decision models, touching on a rare topic of LLM calibration, highlighting an element people may not know or understand.
The author compares new decision models based on #Jev with small local models, finding that APIs are favorable with few examples while results change with more data. Link
The post reports Day 5 of Decision Bench: Microsoft-Decision-1 leads with 96.4% accuracy, while Jev 1.13 trails by just one check, also noting hidden questions and noise comment costs.
The post shares a study on model behavior in multi-answer tasks: models can rank correct options before distractors yet still assign low probabilities to some correct items. The paper evaluates Jev and other models on MetaTool, HotpotQA, and ECtHR. Link