What it does
Asked to bet on a hidden fair die roll, Jev picked a side at near-chance accuracy while claiming high confidence. Noul did much better on the same rolls. Every raw response is in the repo.
Benchmarks & research
Jev Does Not Play Dice
What does a probability model do when the answer is genuinely random? These experiments found out, and the answer is not flattering.
Bookmark: Jev Does Not Play Dice
Keep this in your collection.
Leave a noteWhat would you try with this?
: Jev Does Not Play Dice
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
How you can use it
Before building, gather sample documents that already state exact risk percentages, like inventory shortage forecasts. A developer can borrow this project's test setup to check how the AI scores those known numbers. Your developer can preview the test questions offline without using a paid access key.
Never let the AI recalculate or replace percentages already written in your original business documents. The tests showed that Jev pushes moderate chances to extreme highs or lows. Have your developer directly copy and save the original reported numbers for your app to follow.