Read & learn
Written guides.
Understand Jev, one idea at a time. Walkthroughs, recipes, and writeups — our notes first, the original next.
Explore the Jev ecosystem.
Learn, build, and play with Jev — from deep technical guides to creative experiments.
Watch & learn
Video guides.
See an idea take shape. Tutorials, demos, and deep dives, organized by topic and credited to their creators.
Make & explore
Experiments.
See what builders made with Jev. Games, tools, repositories, articles, and more — traced to their sources.
The directory03 / 03 · Experiments
Find something worth exploring.Experiments
Every category, from games and tools to repositories and writeups, traced to a primary source.
Real-time research powered by
Same ideas.
Different paths.
This is the full registry, not just the featured picks. Figures like speed, cost, and stars are maker-reported or captured snapshots, not JevMade measurements.
409 experiments · showing 121–180
These OpenJev experiments classify sentence pairs, including whether one statement follows from or contradicts another.
SOURCE SCREENSHOTEXPAND ↗
from the source
from classify import ClassifierFactory, OpenJevSettings, TextPairclassifier = ClassifierFactory.create(OpenJevSettings())predictions = classifier.classify([ TextPair("A man is playing a guitar.", "Someone is making music."), TextPair("A man is playing a guitar.", "A man is sleeping."),
A practical guide and prototype for putting Jev triage in front of Herdr-managed coding agents.
SOURCE SCREENSHOTEXPAND ↗
Experiment with a Jev-powered Codenames player and a separate CLI for checking determinism, sensitivity and calibration.
SOURCE SCREENSHOTEXPAND ↗
from the source
questions[word] = style === "compact" ? { type: "noul", instructions } : { type: "noul",
An experimental pharmacy checker that asks Jev the same question five ways and looks for agreement.
SOURCE SCREENSHOTEXPAND ↗
from the source
//! - **Not deterministic.** Identical requests returned 0.03, 0.03, 0.03,//! 0.04, 0.04. This is the entire reason the protocol needs a stability//! gate; the client must not paper over it with caching.//! - **Billing is `input_tokens` only**, with a ~281-token fixed overhead per//! call and state billed once regardless of question count. Batching many
A record of five attempts to find a worthwhile job for Jev—and the reasons each stayed out of production.
SOURCE SCREENSHOTEXPAND ↗
from the source
const r = await client.systemOne({ state: { command }, questions: { harm: noul(question) }, ...(model ? { model } : {}), });
Switchloom experiments with Jev-routed persistent coding agents and publishes pilots where routing failed to beat a single Astra model.
SOURCE SCREENSHOTEXPAND ↗
Jeff 1.0 turns open Gemma and Qwen models into decision scorers, with serving code and an accompanying OpenDecisions test set.
SOURCE SCREENSHOTEXPAND ↗
Jeva is a decision model architecture that encodes options independently to support order stability, variable candidate sizes, precomputed caching, and distinct detection of absent answers.
SOURCE SCREENSHOTEXPAND ↗
SkillScreen is a specialized decision model that evaluates agent skill instruction files to screen out malicious actions before tools are executed.
SOURCE SCREENSHOTEXPAND ↗
Runoff reduces sensitivity to option order by making a shortlist first, then comparing its candidates in every cyclic position.
SOURCE SCREENSHOTEXPAND ↗
VideoJev scores temporal intervals and textual options directly from shared video representations, unifying grounding and multiple-choice question answering without word-by-word token generation.
SOURCE SCREENSHOTEXPAND ↗
This benchmark traces poisoned retrieval context through fixed-choice decisions, sandboxed executions, and persisted filesystem state across synthetic workflows.
SOURCE SCREENSHOTEXPAND ↗
A research experiment testing statistical threshold certification on fast binary language model decisions when deferring to human reviewers is impossible.
SOURCE SCREENSHOTEXPAND ↗
raxIT Labs compares nineteen AI decision models and guardrail services across eight common safety use cases to measure accuracy and cost.
SOURCE SCREENSHOTEXPAND ↗
CaLLM is a post hoc multicalibration procedure designed to mitigate position, verbosity, and model-family biases in pairwise language model evaluations.
SOURCE SCREENSHOTEXPAND ↗
Compare Jev and other judges on a web-research agent’s answers, separating direct-call speed from the wait for scores in LangSmith.
SOURCE SCREENSHOTEXPAND ↗
Read a one-day Jev evaluation that publishes its predictions, test code and response logs, including findings that went against expectations.
SOURCE SCREENSHOTEXPAND ↗
Use a self-hosted GLiFormer model behind a Jev-shaped API, with published comparisons showing where its answers differ from hosted Jev.
SOURCE SCREENSHOTEXPAND ↗
An independent visual driving model chooses among available behaviors in JevPilot, leaving local control code to turn each choice into steering and speed.
SOURCE SCREENSHOTEXPAND ↗
A study of bugs in small programs finds that comparing Jev's answer probabilities works worse than direct questions and execution tests.
SOURCE SCREENSHOTEXPAND ↗
Audit where Jev disagrees with existing yes-or-no rules, then use human checks to explore separate thresholds for each answer.
SOURCE SCREENSHOTEXPAND ↗
NXTG.AI compares Jev's probabilities with Qwen and Claude on contract questions and internal routing tasks, then tests adding retrieved context.
SOURCE SCREENSHOTEXPAND ↗
JevBench compares set-answer AI models on decision quality, confidence, speed and estimated cost, with a public board explaining its scoring rules.
SOURCE SCREENSHOTEXPAND ↗
Sully tests whether Jev can predict appointment bookings from the structure of receptionist calls, without hearing audio or reading transcripts.
SOURCE SCREENSHOTEXPAND ↗
This experiment tests Jev on financial-market prediction and reports that its forecasts perform poorly.
SOURCE SCREENSHOTEXPAND ↗
jev-alpha-bench separates news comprehension from market recall to test whether Jev can predict stock returns.
SOURCE SCREENSHOTEXPAND ↗
This benchmark compares Jev, Gemini, and GPT on structured annotation of São Paulo court judgments.
SOURCE SCREENSHOTEXPAND ↗
from the source
uv syncuv run python scripts/01_coletar.pyuv run python scripts/02_anotar.py jev-latest jev-preview gemini-3.8-flash gpt-5.6-luna... ref-gpt-5.6-sol ref-gemini-3.1-prouv run python scripts/03_referencia.py # lista divergências e auditoria
This project experiments with Jev as an evaluator rather than a task-solving model.
SOURCE SCREENSHOTEXPAND ↗
A position paper arguing that Jev-style models need fuzzy and Hidden Markov primitives before they settle on crisp decisions.
SOURCE SCREENSHOTEXPAND ↗
from the source
git clone https://github.com/dnakhoa/jev-deferred-crispificationcd jev-deferred-crispificationpython3 experiments/run_all.py # E1–E5 + lemma checks, CPU, ~2 min
Jev picked the winner in 64.5% of 10,984 randomized Upworthy headline tests.
SOURCE SCREENSHOTEXPAND ↗
This project tests whether Jev can resolve ambiguous Japanese addresses against Japan Post's KEN_ALL data.
SOURCE SCREENSHOTEXPAND ↗
from the source
npm installscripts/download-data.sh # data/utf_ken_all.csv, data/jigyosyo_utf8.csv を取得...export TYPESAFE_AI_API_KEY=... # Jev を使う場合のみnpm run build # dist/cli.js
jev-lab collects TypeScript experiments on Jev's behavior, accuracy, and response latency.
SOURCE SCREENSHOTEXPAND ↗
A phishing test that pits Jev against Claude Haiku 4.5 on 2,000 emails.
SOURCE SCREENSHOTEXPAND ↗
from the source
cp .env.example .env # then fill in the keys (baseline used: claude-haiku-4-5...uv run prepare_data.py # download + checksum + data/emails.jsonluv run net_floor.py # network round-trip floor to each API hostuv run run_jev.py --limit 10 # smoke test, prints raw answersuv run run_jev.py # pass 1, all 2 000 emails
This reproducible MuJoCo pilot compares Jev, Claude Haiku, and reactive rules on pick-and-place control.
ACTUAL RECORDING12 SEC ↗
from the source
.venv\Scripts\python.exe -m zipfile -e results\evaluation_traces.zip outputs\comparison.venv\Scripts\python.exe demo.py render outputs\comparison\eval_jev_ordinary_1001.jsonl
This game playground compares Jev with other evaluators using explicit states, legal moves, and observable outcomes.
SOURCE SCREENSHOTEXPAND ↗
from the source
flowchart LR state[Current state] --> actions[Legal actions] actions --> model[Model evaluation] model --> choice[Validated choice] choice --> next[Next state]
HackSing puts Jev through 50 tests and publishes the findings in a 52-page Chinese-language report.
SOURCE SCREENSHOTEXPAND ↗
This experiment measures Jev as a lower-cost model router on the RouterArena benchmark.
SOURCE SCREENSHOTEXPAND ↗
Can Jev tell a real credential from an innocent string? This benchmark tests it on file snippets.
SOURCE SCREENSHOTEXPAND ↗
from the source
echo "TYPESAFE_API_KEY=..." > .envuv run python main.py
A test of whether Jev can tell genuine shadcn-ui lint problems from false alarms.
SOURCE SCREENSHOTEXPAND ↗
from the source
node extract-design-system.mjs --repo <lint checkout> # only when the lint repo change...export TYPESAFE_API_KEY=... # never commit itnode run-rule-cases.mjs rule-cases.jsonl --repo <lint checkout> [--limit n]
This notebook compares zero-shot spam judgments from Jev with conventional TF-IDF classifiers.
SOURCE SCREENSHOTEXPAND ↗
from the source
# New output directory: a three-message smoke test, then the full comparison.JEV_CONTEXT_OUTPUT_DIR=experiments/jev-context/rerun \ uv run python experiments/jev-context/evaluate.py --limit 3JEV_CONTEXT_OUTPUT_DIR=experiments/jev-context/rerun \ uv run python experiments/jev-context/evaluate.py
This classifier distinguishes agent-written from human-written pages in the collusion.wiki corpus.
SOURCE SCREENSHOTEXPAND ↗
from the source
ground_truth.py # the per-page agent/human label join (the only "ground truth" logic)baseline_regex.py # content-only floor: regex tells -> shows text alone is blindjev_client.py # minimal TypeSafe Jev client (noul primitive, plain HTTP)qwen_client.py # local Qwen3.8-Flash-Next client (OpenAI-compatible)compare.py # the head-to-head harness + metrics table + JSONL dump
This repository tracks ongoing research into Jev and System One models as a set of Markdown slides.
SOURCE SCREENSHOTEXPAND ↗
This app extracts data from a document according to a supplied JSON schema using parallel constrained decoding.
SOURCE SCREENSHOTEXPAND ↗
A Jev-inspired experiment that makes visual decisions on an iPhone using Qwen3-VL, not TypeSafe's service.
SOURCE SCREENSHOTEXPAND ↗
This study tests Jev as a fast gate for suspicious agent actions in SHADE-Arena.
SOURCE SCREENSHOTEXPAND ↗
from the source
conda create -n shade-arena python=3.11conda activate shade-arenapip install -r requirements-jev.txt # minimal pinned deps; upstream requirements.t...conda env config vars set PYTHONUTF8=1 # task data is UTF-8; Windows defaults to cp125...conda deactivate && conda activate shade-arena
This Rust port runs Jev-style Choice, Score, and Noul evaluations through ordinary language models.
SOURCE SCREENSHOTEXPAND ↗
from the source
use system_one_adapter::{ Noul, ProviderName, SystemOneAdapterClient, SystemOneArgs,};let client = SystemOneAdapterClient::new(true, system_one_adapter::AnswerMode::Probabili... .normalize_probabilities(true);
system-one-gemma adds a scoring head to Gemma 3 270M for calibrated decisions without text generation.
SOURCE SCREENSHOTEXPAND ↗
from the source
from infer import load_trained_model, scoretok, model = load_trained_model("./pretrained-scorer")probs = score(model, tok, state="Patient has chest pain radiating to left arm, ST elevation on ECG", question="What is the triage level?",
These charts place Jev's Thai standardized-exam results alongside those of 110 other models.
SOURCE SCREENSHOTEXPAND ↗
A chess evaluation that gives Jev an explicit list of moves and measures how well it chooses.
SOURCE SCREENSHOTEXPAND ↗
typesafe-oracles tests when a typed Jev judgment is more useful than a conventional language-model call.
SOURCE SCREENSHOTEXPAND ↗
from the source
npm run dataset # build the binary corpus -> .sandboxes/episodes.jsonnpm run run # arm A (Jev) + arm B (Haiku via `claude -p`)npm run run:api # arm C (Haiku via the API — the honest baseline)npm run report # accuracy, confidence separation, calibration, costnpm run run:repeat # 5 independent runs of arm A
This experiment places Jev before each proposed agent action and blocks steps judged unsafe.
SOURCE SCREENSHOTEXPAND ↗
from the source
“Tested TypeSafe’s Jev (no-text, probability-only model) as an AI agent safety monitor....
This experiment recreates Jev-like decisions with Qwen 3.8 27B running on Cerebras.
SOURCE SCREENSHOTEXPAND ↗
from the source
“Built an alternative version of @typesafeai but on @cerebras with Qwen 3.8 27b.
Simila...
Juan Macías tries Jev on Spanish data-protection documents and shares where it succeeds.
SOURCE SCREENSHOTEXPAND ↗
from the source
“He probado @typesafeai y la verdad es que merece la pena muchísimo. Tiene unos casos de...
See how Jev fares in solved poker spots—and how much the wording of the game state changes its play.
SOURCE SCREENSHOTEXPAND ↗
choice
Hero is on the turn and it is hero's turn to act. Which action from `decision.legal_actions` should hero take? · check / all_in
Jev and an OpenRouter model race to score on identical Snake boards.
SOURCE SCREENSHOTEXPAND ↗
Match Jev against a text model in chess, where Jev selects legal moves and its opponent must produce valid notation.
ACTUAL RECORDING12 SEC ↗
Follow a Jev-guided ROS 2 turtlesim route from an early control failure to a corrected four-waypoint run.
SOURCE SCREENSHOTEXPAND ↗
Compare Jev's driving choices when its road description comes from perfect simulator state or a vision model.
SOURCE SCREENSHOTEXPAND ↗
Replay a documented Jev-versus-Stockfish tournament with legal moves, raw responses, complete games, and provisional ratings.
SOURCE SCREENSHOTEXPAND ↗
A Flappy Bird test that grades Jev's decisions and lets you replay the results.
SOURCE SCREENSHOTEXPAND ↗
Jev, briefly
A decision model, not a chatbot.
A chat model generates text token by token and hopes you parse it. Jev never generates a word. Your code sends the state of the world plus typed questions; Jev returns every answer in one parallel pass — typed values with calibrated probabilities and a confidence score your code can trust. About 70–500 ms end to end, $0.042 per million input tokens, output free. Trained with what TypeSafe calls Reinforcement Learning for Calibrated Decisions.
choice Pick one of these.
You define the options. Jev returns the chosen option, a probability for every option, and a confidence. The workhorse of routing, agents, and games — the answer space is always legal, so there is nothing to parse and nothing to hallucinate.
score Rate this on a rubric.
You describe ordered levels — say trivial / normal / critical. Jev returns the level, the probability of each, and a confidence. Scores turn fuzzy judgment (“how severe is this log line?”) into a number-free decision your code can branch on.
noul Is this statement true?
You assert a statement; Jev returns the probability that it is true — a noul. Moderation, verification, guardrails, “does this diff actually fix the bug?”: one question, one calibrated probability.
Official material lives at typesafe.ai and docs.typesafe.ai. JevMade is an independent community registry — not affiliated with or endorsed by TypeSafe AI.
Recurring lessons
Patterns that keep showing up.
One decision per tick
Games, drones, trading bots, and browser agents all converge on the same loop: serialize the state, ask one decisive question, act, repeat. Jev’s latency makes the loop feel instant — the model lives inside the control loop, not outside it.
Confidence as a gate
The answer says what; the confidence says whether to act. The most reliable entries threshold on confidence to route edge cases to a slower model or a human — automation with an honest escape hatch.
Shrink the choice space in code
Makers rarely ask Jev to pick from everything. Local tactics prune 225 gomoku moves to ~40; DOM filters turn a page into an element table; code narrows options, Jev judges within them.
Jev judges, LLMs talk
A recurring split: Jev makes every decision cheaply and instantly, and a small LLM is only invoked when a human-facing string must actually be written. Decision and generation are separate budgets.
Made something with Jev?
Send the link — repo, live demo, post, or video — plus a line on what it does and which primitives it uses. Every entry is verified against its primary source before it ships.
Submit your experimentSign in first so we can reply to you. Submissions are reviewed, not published automatically.