Call quality review
voice_quality.review runs when a call finishes. It produces deterministic findings — slow replies, long gaps, ignored interruptions, false stops, repetition, stock phrases, wrong script, an unheard caller — each naming the setting that would address it, and separately the model's opinion on delivery, labelled as the model's. Findings no control can fix are counted but do not lower the score.
Two kinds of judgement, never mixed#
A deterministic finding comes from measurements taken on the wire: turn_timing on every reply, barge_in on every interruption with how long the voice kept talking, eou events where host turn detection is running. Nothing about it is an opinion, and each one carries the setting change that addresses it.
The model's assessment of delivery is kept beside them and marked as the model's. It is useful — a model is a reasonable judge of whether a reply sounded stilted — and it is not evidence. Presenting the two as one list would make an opinion look like a measurement, which is the specific mistake this separation exists to prevent.
| Finding | Measured from |
|---|---|
| Slow replies | Reply time against 2.5 s target, 4 s slow |
| Long gaps | Silence between turns |
| Ignored interruptions | The voice speaking past 2 s after the caller began |
| False stops | Stopping for something that was not a turn |
| Repetition | The same content returning across replies |
| Stock phrases | Filler language that carries nothing |
| Wrong script | A reply in a script the caller has not used |
| Unheard caller | Speech that started no turn |
The finding that offers no control#
When the model's own first token accounts for MODEL_BOUND_SHARE — 60% — or more of a wait, the finding says exactly that and offers nothing to turn. There is no setting, so naming one would be a lie with a workflow attached.
Those MODEL_LIMITED findings are listed and counted but do not lower score; score_all is the old number that included them. A score that falls for something outside anybody's control trains the person reading it to ignore the score, and a score nobody trusts is worse than no score.
The same reasoning covers the two behaviours that were built, measured and rolled back — respect-word cadence and murmur handling. They are measured (is_murmur, murmur_reactions, murmurs_accepted) so their frequency is known, and they are not offered as controls. Findings a setting cannot fix is the longer argument.
When it runs, and why it does not block the ending#
The review runs at finish(), in the background. end used to wait on the summary and the review — two model calls — inside one 20-second request, and then log "end not recorded" for an ending the engine had recorded perfectly well.
Now finish(defer=True) closes the row, books the cost, marks facts.digest = pending and answers immediately; a background task runs the digest once. So a review appearing a little after the call ends is the design, not a delay.
Reading it as a tuning loop#
- Check whether the finding is deterministic or the model's opinion. Only the first names a setting.
- Check whether it is marked model-limited. If it is, the remaining levers are the model choice and the size of the brief.
- Change one setting in the Voice Lab and place a test call.
- Compare the new call's findings, not the old call's. The settings that ran are snapshotted per call on
call.quality.
One finding sits slightly apart from voice tuning: promised_unbooked, raised when a promise was made on the call with nothing behind it. That is a follow-through problem rather than a delivery problem, and promises made on a call covers the rule that refuses to record a time nobody actually said.
Questions#
Why is the score high when the call clearly felt slow?
Most likely the slowness was model-bound. Those findings are listed and counted but excluded from score, with score_all carrying the older, all-inclusive number. Read the findings list rather than the headline figure when a call felt worse than it scored.
Can I trust the model's comments on delivery?
Treat them as a second opinion, which is how they are labelled. They are useful for things measurements cannot see — whether a reply sounded abrupt — and they are not the basis on which to change a setting. The deterministic findings are.
Does the review run on carrier turn-based calls too?
The measurements it depends on — turn timings, interruption events, end-of-turn events — come from the realtime worker, so a turn-based call has far less to judge. The call record, transcript and summary exist for both paths.