Back to explore

Public benchmark for Jev real decisions: 900 decisions graded by Opus 5

The benchmark includes 900 real Jev decisions graded by Opus 5, with 85.8/100 agreement; misses almost entirely occur where Jev reported low confidence, and anyone can rerun it against their own model.

Open weight builders will need a public yardstick. One went out today: 900 real Jev decisions graded by Opus 5, 85.8 of 100 agree, and the misses sit almost entirely where Jev reported low confidence. Anyone can rerun it against their own model:

· 0 likesOpen on X