Steve from Builder.io tests Jev on automated browser and OS tasks, revealing significant failure rates when operating on raw text representations without visual input. He demonstrates where Jev fails standalone, how a hybrid fallback with multimodal LLMs works, and practical integration patterns like tool selection and priority classification.
Original by Steve (Builder.io)Browser useIntermediate5 min 30 secPublished
Steve’s browser and computer-use harnesses had sharply different completion rates across simple and complex tasks; raw speed did not translate into reliable general computer use.
Compare cost per completed task, because repeated wrong actions and loops can erase a lower per-call price.
The video proposes narrower uses such as tool selection, routing, and a first pass with an LLM fallback; these are not validated universal replacements.
Worth knowing
Results are derived from the author's specific informal harnesses and custom application tests; they are not standardized benchmarks across diverse operating systems.