What it does
Its evaluator keeps reference labels out of the request, then joins them after inference to calculate accuracy and calibration.
Benchmarks & research
Nimble
Nimble compares Jev's typed probabilities with locally trained schema adapters on curated decision datasets.
Bookmark: Nimble
Keep this in your collection.
Leave a noteWhat would you try with this?
: Nimble
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
How you can use it
When testing your own AI, keep the correct answers out of the questions it sees. Start with this project's testing approach: ask the questions first, then compare the results with the saved answers. A developer can adapt that separation for your tests. Matching saved answers on a small test set does not establish how well an AI will handle your whole project.