What it does
JevOss probes reordered options, hidden instructions, added distractions and determinism. Its results describe the particular server being tested. Before applying suite scores to an application, check the model on your own inputs.
Benchmarks & research
JevOss
JevOss is an open-source evaluation toolkit that measures the accuracy, calibration, and robustness of decision models implementing the Jev API.
Bookmark: JevOss
Keep this in your collection.
Leave a noteWhat would you try with this?
: JevOss
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
How you can use it
Try the toolkit when you want to see whether reordering options, adding distractions or planting unwanted instructions changes an AI’s answer. It can also adjust how sure the answers look using examples with known outcomes. Check those adjustments with different examples from the ones used to make them.