Back to explore

How to Evaluate Jev's Accuracy

Discusses that while Jev cannot invent answers outside a predefined schema, it can still choose wrong answers within it, sometimes with high confidence, and raises evaluation questions for low-stakes scenarios.

Jev is cool, but the question I’ve heard most over the past few days is: how do we evaluate its accuracy? 🤔 Jev cannot invent an answer outside a predefined schema, but it still can choose the wrong answer within it, sometimes even with high confidence. For low-stakes,

· 3 likesOpen on X