Before you dive in
What you’ll find in the original
- The replica performs one forward pass, reads next-token logits for allowed letters, applies softmax, and maps those probabilities back to caller-defined options without a generation loop.
- Keep deterministic work outside the model: the Doom implementation uses engine data and BFS for navigation while the model chooses a local action within the current phase.
- Cache the static few-shot prefix only when it matches exactly and size the cache for distinct questions; warm caching cut the reported single-decision path from roughly one second to about 127–173 ms.
Worth knowing
This reproduces Jev's API shape, not TypeSafe's training method, architecture, calibration, or RLCD claims. Results come from small authored test sets and one M4 Mac; the full Doom demo ran slower than real time.