Our summary
Decision models choose among supplied answers rather than writing a reply. Artificial Analysis built this index to compare that choice-making across four kinds of task. Each contributes a quarter of the total: knowledge without invented answers, difficult multiple-choice reasoning, choosing tools and classifying whether text is safe.
The launch comparison also measures fees, response time and how well probabilities match correct answers. Jev leads the decision-model total in that snapshot, while other models lead price or speed. A larger language model scores higher overall but takes longer and costs more under the tested settings.
High confidence is the important warning: the decision models are often less accurate than their probabilities suggest, especially on knowledge and reasoning. Classification and tool choice are stronger areas. The index tests hosted services and fixed tasks, so its changing leaderboard cannot decide which model fits your own work.
Key takeaways
- Read all four task scores. Knowledge and difficult reasoning make up half the index, so a total can hide weaker areas.
- Price, speed and the total score have different leaders in the launch comparison; none alone picks a model for your work.
- Compare probabilities with outcomes you know before treating high confidence as a reason for software to act.