A user tested Jev on the medical QA benchmark igakuQA: 2,000 questions solved in about 80 seconds for $0.04; text-only accuracy was 88.6%, comparable to GPT-4o; using confidence cutoffs further improved accuracy.
話題のjevにigakuQAをとかせてみた。 2000問を80秒ぐらいで解いて0.04ドル。 テキストに限った正解率は88.6%でGPT-4oぐらい。 confidenceが正確と聞いたので、出力させると確かにカットオフによって正解率が上がって面白かった。 医学ドメインでも強そうですね。
During live clinical consultations, an ambient scribe periodically feeds transcripts to Jev, which traverses a medical ontology to classify symptoms, update the differential diagnosis, suggest treatments, and flag red flags in real time.
The author demonstrates how to triage a patient to the right doctor using only TypeSafe AI's Jev model and some question-asking code, noting that no LLM is used during the conversation.
A developer shares building a calorie tracking app with Jev, highlighting its instant experience.
