JevMade field notes / Reproducible evaluation guide

How to guide planning software using plain English rules

This guide explains how to use everyday words to stop planning software from making forbidden moves. It shows how the software translates future steps into simple terms so a decision tool can judge them.

Original by AbdelStarkEvaluation

Bookmark Keep this in your collection.

Leave a noteWhat would you try with this?

Only you can see your notes.

Listen to this guide

JevMade’s plain-English explanation

0:00 /

AI narration

Independent ElevenLabs affiliate. JevMade may earn a commission.

Credits and license

“LeJudge: A cost module you program in English” by AbdelStark. Read the original source.

This expanded guide is an AI-narrated adaptation of the source’s essential explanation, examples and caveats, not a word-for-word reading. The synthetic voice does not imitate the author or imply their endorsement.

License: MIT

MIT License

Copyright (c) 2026 Abdel

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Our summary

This project tests a way to give plain English rules to planning software. Normally, these programs only understand mathematical goals, making it hard to forbid specific actions. This method lets users write simple limits without changing the underlying computer code.

The software guesses hundreds of possible future steps and translates them into basic words, like locations or speeds. Then, it uses Jev—an AI tool that chooses from options rather than writing an answer—to decide if any step breaks the written rules.

This guide is for developers testing software limits. In these tests, the software often failed because its guesses about the future stopped matching reality over time. Also, if the program lacks a specific word in its dictionary, it silently ignores rules using that word.

Key takeaways

Test rules on situations where it is actually possible for the software to succeed.
Check if the program dictionary includes the exact words you use in your written rules.
Turn off confidence filters when testing to see if the decision tool matches a perfect checker.

More guides →See the experiments →