JevMade field notes / Benchmark operations guide

Testing how well AI assistants play Texas Hold'em poker

This guide explains a platform that tests how well AI assistants play poker. It shows how to separate cash games from tournaments and why adjusting for bad luck does not measure overall skill.

Original by ProphetLabEvaluation

Listen to this guide

JevMade’s plain-English explanation

0:00 /

AI narration

Credits and license

“JevPokerBench” by ProphetLab. Read the original source.

This expanded guide is an AI-narrated adaptation of the source’s essential explanation, examples and caveats, not a word-for-word reading. The synthetic voice does not imitate the author or imply their endorsement.

License: MIT

MIT License

Copyright (c) 2026 ProphetLab

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

Our summary

This project is a testing platform and playground for AI assistants playing Texas Hold'em poker. People use it to watch software play against each other, review past hands, or play virtual games themselves. It helps users see how different programs handle complex poker decisions.

The system tracks scores on separate leaderboards for cash games and tournaments. It uses Jev, an AI tool that chooses from options rather than writing an answer, to give advice during games. Players can also add their own private access codes to bring in other AI assistants.

This is useful for people testing AI decision skills. The platform adjusts scores for bad luck during all-in bets, but this does not measure an AI assistant's overall strategy or true win probability. Also, a displayed name does not prove which software is actually running.

Key takeaways

Keep cash games and tournament scores separate because the chip rules and goals are completely different.
Always check the actual software behind a displayed name, since labels do not prove where it came from.
Adjusting scores for bad luck only covers specific situations, not the overall quality of the software's choices.

More guides →See the experiments →