Short answer: Do not judge an AI workflow from a handful of impressive demos. Test it on representative examples with a human-checked answer key, measure different error types, and decide in advance when the system should abstain and ask a person.
Build a useful evaluation set
Collect examples that reflect normal inputs and difficult ones: short and long requests, incomplete forms, unusual wording, mixed languages if they occur in your process, duplicate records, and cases outside the intended task. Remove or protect personal and confidential data according to your organisation’s policies.
Have a knowledgeable reviewer label the expected result. Keep a separate test set that is not used while changing prompts or rules; otherwise a workflow can appear to improve only because it was tuned to the same examples.
Measure more than one accuracy score
- Exact correctness: Was the output right according to the agreed answer key?
- False positive rate: How often did it take an action when it should not have?
- Miss rate: How often did it fail to flag a case that needed attention?
- Abstention rate: How often did it safely send a case to a person?
- Correction time: How much work did reviewers need to repair output?
- Business outcome: Did the workflow reduce total handling time or improve an agreed service measure without harming quality?
Overall accuracy can hide a costly rare failure. If an AI classifies payment exceptions, count incorrect “safe” classifications separately from harmless category mismatches. The cost and reversibility of the action should shape the threshold.
Fictional example: sorting service requests
Imagine an Indian maintenance company receives 500 requests each month. A pilot test of 100 representative cases correctly labels 91; it incorrectly marks 3 urgent cases as routine and sends 6 uncertain cases for review. The team should not report “91% ready to automate” without context: those three missed urgent cases may matter more than six visible manual reviews. The numbers are fictional and demonstrate why error type matters.
For high-impact actions, keep a person’s approval step until testing shows the workflow is reliable enough for a narrowly defined action. Re-test after changing the model, prompt, source documents, or process.
Formula Works’ AI-assisted operations service treats evaluation and exception handling as part of the workflow design. For related metrics, read how to measure time saved.
Written by
Formula Works
Published
Mon Oct 19 2026