Our summary
Decision models choose from supplied answers instead of writing replies. Lijuan Tang and Yuemeng Zheng review 28 early papers to ask what this changes in practice. Their evidence map separates the model's answer format from the surrounding software, so faster or cheaper workflows do not automatically imply more accurate decisions.
The authors compare how studies measure accuracy, time, cost and confidence. Their fourteen-point checklist asks for suitable alternatives, repeated tests, clear data origins and checks on unseen examples. It also asks whether changing option names changes answers, and whether passing uncertain cases to another model actually helps.
This is a review, not a new run of the experiments. Its papers appeared between September 19 and 24, with versions checked through September 25. AI tools helped extract findings, which the authors reviewed. The conclusions concern a young, uneven evidence base and do not settle how every decision model works.
Key takeaways
- Compare against ordinary models that can score the same answer options, not only against models asked to write an answer.
- Check how often confident choices are right on new examples. Test cutoffs separately from the examples used to choose them.
- Measure speed, cost and success separately. Reusing saved answers can change the comparison; one improvement does not prove all three.