Our summary
A classifier sorts information into groups, such as sending a bank message to the right team. Ves Stoyanov asks whether an AI agent can write the sorting instructions in ordinary language, then let Jev do the repeated judgments. People can read and edit those instructions instead of retraining a model.
In the tests, Claude Opus writes and improves the instructions. It either receives all the examples with correct answers or spends a small budget asking a simulated user for labels and rules. Stoyanov reports that the fully labelled classifiers matched or beat a separately trained comparison model on six tasks.
Asking about hidden rules helped routing; asking for uncertain examples worked better when the answers were already suggested by the text. No strategy won everywhere, and extra questions sometimes hurt. These are the author's single or few-run results with a simulated user, not proof of the same gains with real customers.
Key takeaways
- The agent builds and checks the instructions; Jev handles the repeated sorting.
- A budget worth about 50 labels can also be spent asking about rules; the right choice depends on what is missing.
- Readable rules can be edited, but the reported comparison does not establish a universal winning strategy.