Reviewing calls as a routine
Read four things each week: how calls ended and who ended them, which calls were never conversations at all, promises with no booking behind them, and reply latency. Each points somewhere specific — attribution at a line or a model problem, unbooked promises at the wording of the instructions, latency at a floor no setting moves.
The four fields, in reading order#
| Field | What a pattern in it means | Where the fix lives |
|---|---|---|
| Who ended the call | Endings are attributed to the caller, the model, the owner, the engine, the carrier or the agent. A cluster on any one of the middle four is a fault, not a preference | Engine or carrier endings point at the line and the provider; model endings point at the conversation layer |
| Whether it was a conversation | A call that rang and was not answered is different from one that never rang at all, and they have their own words on the record | Never rang means the dial, the dispatch or the worker failed; rang and unanswered is a fact about the person called |
| Promises against bookings | A promise made in a call with nothing scheduled behind it is a recorded finding in its own right | The instructions, and the follow-up rules — not the caller |
| Reply latency | Median reply time across the week, against the target and the slow threshold | Prompt size, greeting warm-up and the model. The best measured call sat at a 3.3-second median reply, which is the floor rather than a target to beat |
Read them in that order because they narrow. Attribution tells you whether anything is wrong; the conversation word tells you whether the call was ever a conversation to judge; promises tell you whether the business kept what it said; latency tells you how it felt to the person on the other end.
The review, stage by stage#
- Trigger — a weekly slot, or a report that something on the phone is not right.
- User event — somebody opens the call list for the period rather than reacting to one memorable call.
- Authentication and workspace resolution — the review happens inside one workspace; every call listed belongs to it.
- Ingest — the calls, their outcomes and their attributions are already recorded; a review reads rather than reconstructs.
- Canonical record — each call carries its ring time, direction, line, outcome, attribution and, where there was a conversation, its transcript and summary.
- Classification — group by attribution first, then by outcome. One bad call is an anecdote; five with the same attribution is a finding.
- Knowledge, memory and rules — a recurring question the voice could not answer is a missing knowledge source, not a model failure.
- Autonomy and approval — a change to what the voice says is a change to instructions, and goes through the tuning cycle rather than being edited mid-week.
- Action — one change at a time, so the next week's numbers mean something.
- Result — the finding is either fixed, or recorded as one a setting cannot fix.
- Relationship and timeline — calls that produced a lead, a case or a follow-up are already attached to their person; the review checks that they were, not that they exist.
- Audit and usage — cost is booked per call, so an unusual week shows up as spend as well as behaviour.
Findings a setting cannot fix#
Roughly three in five of the findings from the recorded gap review were model-bound: behaviour that comes from the model's own judgement rather than from anything configurable. Recognising one early saves a week of adjusting settings that were never the cause.
- First-token latency
- The gap before the first word is the model's, plus end-of-turn detection. Shortening the prompt helps measurably up to a point, and then the floor is the floor.
- Replies that ramble
- Answered by instructing one thought and one question per reply. That change moved the median reply from the twenties-to-forties down to about nineteen words, and removed stacked questions entirely.
- Re-introducing itself mid-call
- Fixed by telling the model the opening has already been spoken, which removed re-introductions altogether.
- A session that would not start
- Retried once on the same model and once on a fallback before the call is given up, and the restart is on the record — so this is visible rather than mysterious.
- A price spoken that should not have been
- On the realtime path the rule lives in the instructions, so a breach is detected rather than prevented. The lines that priced something are counted, which is what makes the review possible at all.
What not to conclude#
- Do not read a summary of a call that was barely a call. A call with fewer than two caller lines and fewer than five caller words is never summarised, precisely because a summariser handed almost nothing once invented an afternoon that did not happen.
- Do not treat a refusal as a failure. A line that was switched off or closed answers with a spoken message and ends; the outcome says which, and the caller heard something rather than silence.
- Do not chase a single slow call. Latency is a distribution; a median across the week is a fact and one call is weather.
- Do not change two things at once. The next review cannot attribute the difference, and the second change usually turns out to have been unnecessary.
- Do not review only the calls that went badly. A week of calls that ended correctly, with promises booked and latency inside the target, is the baseline everything else is judged against.
What the review should produce#
- One change to make, or an explicit decision that this week's findings were model-bound and no change is warranted.
- Any missing knowledge source, added — a question that recurs is cheaper to answer once than to escalate weekly.
- Any follow-up promised and not booked, chased by a person, because the customer is waiting on the promise rather than on the record of it.
- A note of what was normal, so that next week's unusual is recognisable.
Questions#
How many calls make a pattern?
Enough that the same attribution or outcome appears more than twice for different callers. Two calls sharing a fault can be one caller with a bad line; five sharing it is the line, the provider or the instructions.
Should the review listen to recordings?
There are none to listen to. Call recording is not enabled on the live carrier, so a review reads transcripts, summaries, outcomes and attributions — which is enough for every finding described on this page.
Is a call that ended at the length ceiling a problem?
Not by itself. Calls are no longer cut when a model session reaches its own limit; the product's own ceiling is thirty minutes. A call reaching it is worth reading, because half an hour on the phone usually means something was not resolved.