JevMade hello@JevMade.com
← Back to experiments

Benchmarks & research

jev-tmmluplus-eval

Puts the Traditional Chinese TMMLU+ exam to Jev, one four-way question per item, and scores it the way the official leaderboard does.

Source screenshot of jev-tmmluplus-eval
SOURCE SCREENSHOT · source ↗ · captured 2026-09-24Full screenshot ↗

What it does

The four options are the response format, so a stray answer is impossible and every question parses cleanly.

Maker-reported (not independently measured by JevMade): 77.57 (ievals category macro) · 78.66 pooled · 19,646 questions · 66/66 subjects · 0 API errors · $0.42, 91 seconds at --concurrency 64

Primitives
choice
Platform
Python
Added
Project created

Source checked 2026-09-24 — opened the GitHub repository directly.