JevMade

Sign in
← Back to experiments

Benchmarks & research

Retriever-Free Retrieval (DecisionRAG)

DecisionRAG skips the search index and lets Jev browse a document collection like a library, keeping or dropping topics, then pages, then passages.

Source screenshot of Retriever-Free Retrieval (DecisionRAG)
SOURCE SCREENSHOTFull screenshot ↗

What it does

The Cornell and ASAPP authors report beating dense retrieval with an 8B reranker by 13.8 points of top-1 recall on MultiDoc2Dial and 17.4 Hit@1 points on Natural Questions, in 0.7–1.8 seconds per query. It only works where documents are already grouped into a hierarchy, and a branch dropped early is never revisited.

How you can use it

You can borrow this browsing approach to search large document collections without traditional search indexes. Your developer organizes your text into trees, like topics, documents, and paragraphs. An outside AI service called Jev then looks at branch summaries, picking which paths to keep and which to ignore until reaching individual passages.

Because this research preprint shares no ready-to-install code, your developer must build the hierarchy and navigation steps themselves. A major limit is that dropped branches are never revisited. If an early summary is unclear or misread, the right answer can be permanently missed.

Maker-reported (not independently measured by JevMade): Authors' results without task-specific training: +13.8 R@1 on MultiDoc2Dial and +17.4 Hit@1 on Natural Questions over dense retrieval (Qwen3-Embedding-8B) with Qwen3-Reranker-8B. · Main table, DecisionRAG (Jev) / with dense candidates added: MultiDoc2Dial R@1 47.4 / 48.5; Natural Questions Hit@1 65.4 / 66.4. GPT-5 reranking of the dense top 20 scored 47.5 and 60.8. · Measured per query: 0.68–1.76 s median latency and 0.22–2.33 cents API cost. GPT-5 reranking cost 1.20–1.90 cents, so DecisionRAG was not cheaper on MTRAG or Natural Questions. · With the hierarchy fixed, GPT-5 writing out navigation decisions was 34–145× slower and 21–25× more expensive, with no consistent accuracy gain. · Judging candidates in one shared context added 7.7 R@1 on MultiDoc2Dial; the gain levels off at about 50 candidates per request. Choice reranking is capped at 255 options. · Natural Questions: 2.68M paragraphs in 1,000 clusters; keeping 15% of clusters matched flat scoring of all 108,626 page titles at a fifth of the cost. · A Qwen3-8B decision model trained on MultiDoc2Dial data reached 60.7 R@1, 5.7 points above the strongest published task-trained retriever compared.

Primitives
noul, choice
Platform
Jev 1.13 API
Added

Keep this for later

Sign in to bookmark experiments, guides and videos, and keep notes only you can see.

Continue to sign in

We’ll bring you back to this listing.