Our summary
The author tests ways to measure how well an AI tool makes decisions. Instead of asking the software to write a sentence, the experiment uses Jev, an AI tool that chooses from options rather than writing an answer. This helps test decision skills in a controlled environment.
The setup gives the AI a text description of the current board and whose turn it is, along with a complete list of allowed moves. The software must pick one move from that list. This matches the tool's design better than trying to make it generate words.
This approach is useful for developers who need software to pick from a set menu, like sorting incoming messages. The chess rankings remain exploratory because the number of games played is small. The author reported the scores and prices, which may change over time.
Key takeaways
- Give the AI a text description of the board and a strict list of allowed actions.
- Picking from a menu works much better for this tool than generating text from scratch.
- The chess scores are interesting, but the small number of games makes the results exploratory.