Our summary
A busy blog can receive help requests, corrections, thanks and spam in the same queue. Flavio Copes shows how Jev can sort these comments into categories chosen by the site owner. The important work is explaining what each category means, then checking whether the answers are useful.
One question chooses a single label, including an other option for comments that do not fit. Separate yes-or-no questions can add several topic tags, or none. An ordered scale describes how urgently a reply is needed. Each comment keeps its own identity, even when several requests finish in a different order.
Copes recommends testing real comments that a person has already labelled, using a fixed model version. Check how often kept answers are right and how many comments remain unlabelled at each cutoff. His example probabilities and cost estimate are illustrations, not measured accuracy. Uncertain answers go to a person before automatic action.
Key takeaways
- Explain the difference between nearby labels, such as a reader's broken setup and an error in the article. Add examples when tests reveal repeated confusion.
- Send only the context each question needs. Ask independent questions about one comment together, and limit how many comments are processed at once.
- A yes-or-no probability is not the same measure as certainty about a winning label. Test each cutoff on your own comments before using it.