Our summary
Software can receive an answer from a fixed list even when the message gives no evidence for any option. Safouane Chergui explores this problem through small Jev tests. His seven lessons explain how to write clearer questions, allow an answer for missing information and decide which results need a person.
Each question sees the shared input and its own instructions, not another question's answer. Chergui combines separate judgments in code, adds descriptions to confusing choices and examines probabilities across the whole answer set. He reports that clearer descriptions improved correct labels from 82.1% to 88.3% on 308 reserved banking messages.
Those figures describe one small reported test, not Jev's accuracy everywhere. Changing options can change both probabilities and the cutoff needed for automatic handling. An average rating can hide disagreement between opposite ends of a scale. Read the distribution, test on separate examples and keep exact arithmetic and date comparisons in code.
Key takeaways
- Offer a way to answer that the text does not say. A fixed answer list stops invented labels, not wrong choices from that list.
- Put information needed by several questions in their shared input. Ask together only when no question needs another answer or newly gathered evidence.
- After changing a choice's wording or options, test the cutoff again. Count both mistakes and the share of messages left for a person.