JevMade hello@JevMade.com
← Back to experiments

Benchmarks & research

S1Rank

Tests Jev as a search reranker, and asks whether its confidence can tell you which queries are worth spending more compute on.

Source screenshot of S1Rank
SOURCE SCREENSHOT · source ↗ · captured 2026-09-24Full screenshot ↗

What it does

The routing idea did not work. It also found that 52% of probabilities change across byte-identical requests.

Maker-reported (not independently measured by JevMade): BM25 nDCG@10 50.6 / 48.0 / 59.5 / 32.2 / 67.9 across DL19, DL20, TREC-COVID, NFCorpus, SciFact · Jev joint pointwise 73.7 / 71.7 / 86.5 / 38.3 / 80.5 at 1 call per query, $0.57 per 1k queries · 52% of probabilities change across byte-identical requests

Primitives
noul, choice, score
Platform
Python
Added
Project created

Source checked 2026-09-24 — opened the GitHub repository directly.