Back to explore

Comparing Jev vs Codex Luna as Graders on Code Taste Tasks

The author ran a toy pipeline decomposing text-based rubrics into structured answers to compare Jev and Codex Luna as graders on FrontierCode-style code taste tasks.

ran a toy pipeline comparing jev vs codex luna as graders on FrontierCode style "code taste" tasks: decomposed text-based rubrics into structured answers (e.g. are all changes necessary? do all changes consistently use the abstractions required by the task?) -> run both models in

· 0 likesOpen on X