Benchmarks & research
JevBench measures the probabilistic coherence of typed decision models across 50 metamorphic relations without requiring gold labels.
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
A high coherence score does not establish accuracy. A model could ignore the input and give uniform answers, scoring well for coherence while remaining at chance accuracy. This JevBench is also a different suite from Benchmark Heaven’s accuracy benchmark of the same name.
Install the Python package and run its command-line check against your model’s server address. JevBench sends a fixed series of tests and reports how closely the probabilities follow basic probability laws.
For example, a specific option should not be more likely than the broader category containing it. The suite changes questions slightly and compares the answers to check whether they fit together logically.