Benchmarks & research
Researchers integrated Jev into five AI agents across household and mobile tasks, finding that while individual decisions became faster and cheaper, overall task success often declined.
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
Across 4,500 task executions, extra reasoning steps and longer interactions frequently used up the savings from individual Jev decisions. Clearly specified tasks worked well, but ambiguous goals caused difficulty. In mobile environments, Jev also failed to recover from early mistakes.
Make the agent’s goal and the rule for finishing a task explicit. Jev works best here when it can match a well-understood situation to a specific next step.
Do not leave recovery entirely to Jev. Have another part of the program check that each action worked before the agent proceeds, since it may struggle to correct its own mistakes.