Benchmarks & research
Jeff 1.0 turns open Gemma and Qwen models into decision scorers, with serving code and an accompanying OpenDecisions test set.
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
Geuter, Cheng and Alvarez-Melis report 84.4 on OpenDecisions versus Jev’s 80.1, and 81.5 on JevBench v1.7. These are author-run comparisons. The server accepts 8,192 input tokens and up to 16 options, evaluating each question separately. Its default mode pushes middling yes/no probabilities to 0.199 or 0.801 for JevBench scoring; --no-commit keeps the underlying values. Apache-2.0 serving code does not replace the checkpoints’ separate backbone licenses.
Ask a developer to try one released checkpoint with the supplied server and your own labelled examples. It can choose a category, rate an item or answer yes or no. Disable the default probability adjustment with --no-commit before using its uncertainty estimates to decide what your app should do.