Our summary
A customer message can fit more than one department. Hafiz Riaz builds a system that asks Jev where to send it, then lets code decide whether to act or ask a person. The model's confidence describes its uncertainty, not a promise that this answer is correct.
Each department has its own confidence cutoff, with a higher bar for refunds. Missing or low confidence sends a ticket to a review queue, a list people must work through. Reviewers accept or change the suggested department. Automatic routes also need a recorded outcome when the ticket closes.
Riaz tests 60 tickets he wrote and labelled himself, so the results illustrate a method rather than prove performance on new customer messages. His fallback cutoff of one can still admit an unknown department at exactly one. An explicit rule sending unknown departments to a person is safer.
Key takeaways
- Set cutoffs according to the cost of a wrong route, rather than using one number for every department.
- Record outcomes for automatic routes as well as reviewed tickets; checking only uncertain cases gives a biased picture.
- Handle missing confidence and unknown departments explicitly. A review queue needs people to clear it, not just a counter.