Our summary
Flying Higher AI built a test support desk from ten fictional policy documents and twenty customer questions. The documents went into Qdrant, a service that finds related passages. An initial n8n workflow left one AI agent to handle the decisions and write replies, but it mishandled questions including an injury report.
The author split those jobs into separate Jev steps. Choice identified the issue, Score rated urgency and Noul asked whether the retrieved documents contained an answer. A separate AI agent drafted replies when that check passed. Otherwise, the workflow alerted people through Slack and recorded the outcome in a spreadsheet.
Every case meant for a person reached one, but the document check rejected too much. Only eight of twenty outcomes matched the author’s expectations. This small experiment shows how separate decisions make errors visible; it does not show a finished support system or prove that stricter gates are always safer.
Key takeaways
- Finding relevant passages and deciding whether they answer a question are different jobs.
- Test ordinary questions as well as risky ones: a strict check can create unnecessary human work.
- Prepare expected outcomes first so a working-looking automation can be checked against something concrete.