What it does
It also shows where the bound stops working: Jev returns exactly 1.0 on more than half its answers, and some of those are wrong.
Benchmarks & research
jev-certify
Turns Jev's probabilities into a routing threshold with a provable bound on how many queries get silently misrouted.
Bookmark: jev-certify
Keep this in your collection.
Leave a noteWhat would you try with this?
: jev-certify
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
How you can use it
A developer can use this code to sort incoming messages. The app connects to an outside AI service called Jev using an access key. Math rules limit how often the AI sends a message to the wrong place.
You can test the tool by grading about forty AI answers yourself. The system cannot catch every mistake. The AI sometimes claims it is completely sure when it is actually wrong. Sudden changes in what people ask will also break these rules.