Back to explore

Jev model evaluation on BANKING77 and adversarial injection analysis

The referenced blog reports Jev zero-shot accuracy of 81.2% on BANKING77, below MiniLM+linear head's 89.9%; injected text moved prediction probability from 0.76 to 0.48, suggesting a potential security risk.

⚠️ Caveats: • Vendor evals score agreement with GPT-6 Astra + Fable 5.1, not ground truth • BANKING77 (@manojlds): MiniLM + linear head 89.9%, Jev zero-shot 81.2% • Injected text moved a block prob 0.76 → 0.48 https://stacktoheap.com/blog/2026/09/22/jev-attack-of-the-classifiers/…

Jev model evaluation on BANKING77 and adversarial injection analysis 1
· 1 likesOpen on X