Jev beats Gemini 2.5 Flash Lite on classifier eval quality and speed
Malte Ubl ran TypeSafe AI's Jev against an existing classifier eval, where it saturated the eval on quality and was 6x faster than Gemini 2.5 Flash Lite.
The author compares Jev, Semif (qwen3.5-4B extracting logits from a classification head), and a self fine-tuned BERT-like model in a scenario of interest, finding Jev outperforms Semif but is slightly worse than the self-trained model.
Recently I test the performance among Jev, Semif(qwen3.5-4B extract the logits from classification head) and my own fine-tuned bert-like model in one my interested scenario, Jev outperforms the Semif but a bit worse than mine