JevMade

Sign in
← Back to guides

JevMade field notes / Research writeup

Testing an AI choice tool on a public chess scoreboard

This guide explains an experiment using chess to test Jev, an AI tool that chooses from options rather than writing an answer. The author compares its choices against other AI assistants on a public scoreboard.

Original by Maxim SaplinEvaluation

Listen to this guide

JevMade’s plain-English explanation

0:00 /

Our summary

The author tests ways to measure how well an AI tool makes decisions. Instead of asking the software to write a sentence, the experiment uses Jev, an AI tool that chooses from options rather than writing an answer. This helps test decision skills in a controlled environment.

The setup gives the AI a text description of the current board and whose turn it is, along with a complete list of allowed moves. The software must pick one move from that list. This matches the tool's design better than trying to make it generate words.

This approach is useful for developers who need software to pick from a set menu, like sorting incoming messages. The chess rankings remain exploratory because the number of games played is small. The author reported the scores and prices, which may change over time.

Key takeaways

  1. Give the AI a text description of the board and a strict list of allowed actions.
  2. Picking from a menu works much better for this tool than generating text from scratch.
  3. The chess scores are interesting, but the small number of games makes the results exploratory.

The game results, prices, and rankings are reported by the author and can change on the live scoreboard.

Article · Original published

Read the original guide Opens the author’s site in a new tab.

Your note

Only you can see this. Clear the text and save to remove it.

Keep what you save

Sign in to keep bookmarks and private notes on every device. We email you a link; there is no password, and the same link creates your account.