Jev-as-a-Judge with Comet Opik

What it does

Evaluates synthetic refund-support traces with four typed criteria and records auditable results as an Opik experiment.

How you can use it

For your refund helper, ask a developer to try the tutorial's ten made-up chats. Check each answer for invented policies or false claims that a refund happened. Start with the demo that needs no AI connection. Its scores are made up to show how the checks work, not to measure how well an AI judges.

A developer can then connect the real AI service and save each check's score. This sends the test chats to the service. The tool reviews answers only; it cannot refund money or alert a person on its own.