This benchmark traces poisoned retrieval context through fixed-choice decisions, sandboxed executions, and persisted filesystem state across synthetic workflows.
Wu, Chang and Yan trace 1,659 Jev 1.13.0 calls across 20 tasks in two synthetic workflows. Standalone poisoned evidence caused five security violations in 60 draws; a carrier/payload pair caused one in 60. Outer-instruction tampering produced 60 wrong actions, a separate result. A source gate withheld clean actions, while a scope-aware repair restored 12 of 16. Statistical contrasts did not remain significant after Holm correction; the released code supports investigation, not a universal attack-rate estimate.
How you can use it
Borrow the test’s full chain when assessing an automated decision: what evidence was retrieved, which option won, what action ran and what changed on disk. The released benchmark includes an offline smoke test. Keep wrong choices separate from actual security violations, and test whether a proposed defense also blocks legitimate work.
