Back to explore

Jev vs Gemini: A/B Test on 300 Customer Requests

The author tested Jev and Gemini on 300 customer requests. Asking more questions cut the agent from ~7 model calls to 3 and made both models more accurate. Jev also showed a nearly 4x better trade-off when filtering wrong answers.

1/7 I tested Jev and Gemini on 300 customer requests. Asking MORE questions cut the agent from ~7 model calls to 3, and made BOTH models more accurate. But Jev also had a nearly 4x better trade-off when filtering wrong answers. Then Gemini hit a limit that had nothing to do

Jev vs Gemini: A/B Test on 300 Customer Requests 1
· 1 likesOpen on X