JevMade

Sign in
← Back to guides

JevMade field notes / Explainer with benchmark

See how open decision models turn one pass into probabilities

Francesco Cerizzi explains how open decision models work, then tests seven designs, Jev 1.14 and OpenAI's Decisions API on the same 496 decisions.

Original by Francesco CerizziEvaluation

Listen to this guide

JevMade’s plain-English explanation

Expanded guide unavailable: the complete source text could not be retrieved.

0:00 /

Our summary

Francesco Cerizzi wrote this Medium post, published on 4 November 2026. It explains what a decision model is and how open ones on Hugging Face read a state plus typed questions and return one probability per answer, then compares seven designs with Jev 1.14 and OpenAI's Decisions API.

On his 200 routine decisions the four larger open models were 93% to 98% right and answered in 106 to 194 ms on one GPU, similar to Jev 1.14 and the Decisions API. On long records the spread was wide, from 53.6% to 97.0%, with the two models that read the record in both directions leading.

It is one author's test on his own 496 decisions, run on rented GPUs and AWS Lambda, so latency includes network time for the hosted services. Calibration did not always carry over: Jev's calibration error rose on long records. The author points to the Jev Decision Index for choosing a model.

Key takeaways

  1. Use a decision model wherever your code needs one closed answer with a probability, such as routing, guardrails or triage.
  2. Measure calibration on data that looks like yours, because a temperature fitted on one kind of record may not carry over to another.
  3. Check long, rule-heavy records separately, because that is where the designs differed most.

This is an independent author's benchmark, not a vendor claim. JevMade did not reproduce it. Jev's weights are closed, so the author describes it from TypeSafe's documentation and outside analysis. The author says to use the community Jev Decision Index for choosing models.

Medium · Original published

Read the original guide Opens the author’s site in a new tab.

Keep this for later

Sign in to bookmark experiments, guides and videos, and keep notes only you can see.

Continue to sign in

We’ll bring you back to this listing.