Back to explore

Jev Test Ranking: 96% of Failing Tests in Top 10%

On 113 Defects4J bugs, the Jev model ranks test suites so that 96% of failing tests appear in the top 10%, outperforming BM25 and GPT-5.4 nano to fail builds faster.

Question: given a patch, can you rank a test suite so the tests that fail show up first? On 113 Defects4J bugs, how often is a failing test in the first 10%? BM25 → 73% GPT-5.4 nano → 92% Jev → 96% fail your builds quicker instead of waiting :)

Jev Test Ranking: 96% of Failing Tests in Top 10% 1
· 0 likesOpen on X