Short answer: Start with an assistant that helps staff find approved information or draft replies. Let it handle only questions with clear, current answers; route complaints, account changes, safety issues, and uncertain cases to a person.
The aim is not to make every conversation autonomous. It is to reduce repetitive searching or drafting while preserving ownership of customer outcomes.
Set response boundaries
- Connect answers to approved, current help content or internal procedures.
- Require the assistant to say when it cannot find a supported answer.
- Keep refunds, account access, payment changes, commitments, and sensitive decisions behind human approval unless the business has explicitly designed and tested a safe process.
- Make handoff easy and include the conversation context for the human agent.
- Record which source supports an answer where that is technically available.
Measure quality with speed
Track answer correctness, resolution rate, repeat contact, escalation rate, reviewer correction, response time, and customer feedback. Sample conversations regularly. A short average response time is not a win if customers need to contact the company again to fix the answer.
What research can and cannot tell you
A study of 5,172 customer-support agents found a 15% average increase in issues resolved per hour with an AI conversational assistant, with effects varying by agent experience. This is a result in one company’s support setting, not a guaranteed gain from deploying a chatbot or a prediction for another organisation. Published study
Fictional scenario: A home-appliance service desk gets repeat questions about warranty documents. An assistant searches the approved policy, drafts a response, and offers the source paragraph. If a serial number is missing or the request involves an exception, it creates a handoff with the details already collected. The scenario shows a possible design; it is not a real Formula Works deployment.
Before launch, test the assistant on real but privacy-safe historical conversations and ask support staff to rate both correctness and usefulness. For a broader measurement method, see how to measure AI accuracy and human-in-the-loop AI.
Written by
Formula Works
Published
Fri Oct 23 2026