JevMade

Sign in
← Back to experiments

Benchmarks & research

QApilot: Jev screen classification

QApilot’s screen-classification study found Jev cheaper and faster than GPT-5.1, but substantially less accurate on its 371 mobile-app screens.

Source screenshot of QApilot: Jev screen classification
SOURCE SCREENSHOTFull screenshot ↗

What it does

Bonsai describes the screenshots in text before Jev sees them, but that step’s latency is absent from the timings. The maker reports 59.8% accuracy for Jev against GPT-5.1’s 93%; the ambiguous “others” category caused trouble. Different average confidence on right and wrong answers does not, by itself, make a reliable error detector.

How you can use it

The first AI reads an image and describes the screen. Jev then uses that text to choose a screen category. If you adapt this two-model approach, measure the entire chain: the description step can add both time and mistakes, which a test of Jev alone would miss.

Maker-reported (not independently measured by JevMade): Surendranath Reddy Jillella / QApilot reports: Jev median latency was 232 ms compared to GPT-5.1's 4781 ms across 371 screens. · Surendranath Reddy Jillella / QApilot reports: Total cost for Jev was $0.014 versus $2.14 for GPT-5.1. · Surendranath Reddy Jillella / QApilot reports: Jev achieved 59.8% overall accuracy, compared to 93% for GPT-5.1. · Surendranath Reddy Jillella / QApilot reports: Jev's average confidence score on the 'others' category was 0.84 when correct and 0.54 when wrong.

Primitives
choice
Platform
Hashnode
Added

Keep this for later

Sign in to bookmark experiments, guides and videos, and keep notes only you can see.

Continue to sign in

We’ll bring you back to this listing.