What it does
The tools are mocked, and the evaluation can run with mock models too.
Benchmarks & research
jev-eval-agent
A benchmark of how Jev and a language model choose among 100 tools for a personal assistant.
Bookmark: jev-eval-agent
Keep this in your collection.
Leave a noteWhat would you try with this?
: jev-eval-agent
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
How you can use it
For a personal assistant project, start with the saved flight-booking tests. The dashboard compares an assistant offered 100 actions with one offered a single action chosen by another AI. Check which took fewer steps and made fewer wrong choices. The flights and messages are fake, so you can study the method without making real bookings.