What it does
A yes-or-no judge decides at 0.5 and cannot abstain; a category list can include an unknown answer. These judges provide no generated explanation, Score output or full-trace review, and each sends one question.
Integrations
MLflow — TypeSafe typed judges
Evaluate application answers in MLflow with Jev-backed yes-or-no or category judgments, retaining probability information alongside each result.
Bookmark: MLflow — TypeSafe typed judges
Keep this in your collection.
Leave a noteWhat would you try with this?
: MLflow — TypeSafe typed judges
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
How you can use it
A developer can use this tool to test your app's answers. An access key connects the test to the outside TypeSafe service. The service reads a user question and your app's reply. It then returns a simple yes or no. You can check these decisions and the numbers showing how sure it is.
A basic yes-or-no test must always pick a side. If you expect uncertain answers, ask your developer to provide a list of choices. Make sure that list includes an unknown option. The outside service reads the exact text you send. It will see any private details in those messages.