What it does
The four options are the response format, so a stray answer is impossible and every question parses cleanly.
Benchmarks & research
jev-tmmluplus-eval
Puts the Traditional Chinese TMMLU+ exam to Jev, one four-way question per item, and scores it the way the official leaderboard does.
Bookmark: jev-tmmluplus-eval
Keep this in your collection.
Leave a noteWhat would you try with this?
: jev-tmmluplus-eval
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
How you can use it
Choose which subjects from the 66-topic Traditional Chinese exam you want to test. A developer can run the test tool for you using an account key from TypeSafe. The tool gives the model each question and forces it to pick one of four answers directly.
Because the system only sends back a chosen letter and chance numbers, nobody has to clean up chatty writing or long explanations. Remember that this tool only grades the built-in exam questions, not your own custom quizzes or written essays.