Reply latency on a call
Latency is timed on the wire: from the moment the caller's speech stops to the first audio coming back. The target is 2.5 seconds and 4 seconds is marked slow, but the realtime path has a floor — the model's own first token plus its end-of-turn detection — and the best measured call sat at a 3.3-second median. When the model is most of the wait, the review says so and offers no dial to turn.
Measured on the wire, not from the transcript#
The worker's TurnClock starts when the caller's user state leaves speaking and stops when the agent state enters speaking — last word in, first audio out. Every reply posts a turn_timing event with that number.
The obvious cheaper approach is to subtract the timestamps on two transcript rows, and it is wrong in a way that flatters the system enormously. The first real call measured that way showed "8 ms" beside replies the caller had waited seconds for, because a transcript row is written when a sentence is *complete*, not when it started. Any latency figure taken from transcript rows is measuring transcription, not conversation.
The floor, and where it comes from#
| Model or condition | First token | Note |
|---|---|---|
| Gemini 3.1 Flash Live | 0.6 s | The default model, measured on the host |
| 2.5 native audio | 1.2–2.2 s | Chosen when mid-call notes matter more than a second per reply |
| 7,500-character instructions | 1.2–1.8 s | Prompt size is on the record as session.instructions_chars |
| 9,600-character instructions | 2.3–3.4 s | The same model; 2,100 more characters |
| Best measured call, median reply | 3.3 s | First token plus end-of-turn detection, end to end |
Two things fall out of that table. The first is that the realtime floor is not one to two seconds and nobody should promise it: end of turn has to be *detected* before generation can start, and the detection is not free. The second is that the prompt is a latency control — the gap between 7.5k and 9.6k characters is a second and a half of a caller's life, per reply.
That is why the call brief is built against character budgets rather than assembled from whatever is available: knowledge at 4 facts and 2,000 characters, memory 700, the contact block 600, a human voice profile under about 2,500.
A slow reply against a slow model#
voice_quality.review judges against TARGET_RESPONSE_MS (2.5 s) and SLOW_RESPONSE_MS (4 s), and then does something less usual: it asks how much of the wait was the model's own first token. Past MODEL_BOUND_SHARE — 60% — the finding says the model is the cause and offers no setting, because there is no setting.
Those MODEL_LIMITED findings are listed and counted but do not lower score; score_all is the old number that included them. A tuning score that drops for something no control can change teaches the person tuning it to ignore the score.
Check whether the finding is marked model-limited.
Result If it is, no amount of Voice Lab work will move it. The remaining levers are the model choice and the prompt size.
If it is not, look at the reply length and the instruction size before the exotic settings.
Result Shorter replies and a smaller brief move first-token time measurably; most other knobs do not.
Re-run and compare
turn_timingon a fresh call, not the same one.Result The resolved settings are snapshotted on
call.quality, so each review judges the configuration that actually ran on that call.
What did not cost anything#
The 2026-09-06 change that shortened replies is the useful counter-example. Asking for one thought, one question and under 25 words moved the median words per reply from 23–40 to 19 and removed stacked questions entirely, at a reply median of 2.9 s and no latency regression. Speaking less is not slower; it is very often how a call starts feeling fast.
Questions#
Why is my call slower than the 0.6-second first-token figure?
Because first token is not a reply. A caller waits for end-of-turn detection, then the model's first token, then enough audio to be audible. The 3.3-second median on the best measured call is the honest end-to-end number for the realtime path.
Does a longer knowledge base slow every call down?
Only the part of it that reaches the prompt, which is capped: four facts and 2,000 characters. That cap exists because the measured difference between a 7.5k and a 9.6k prompt was more than a second on first token, and knowledge is the easiest thing to let grow without noticing.
Can I see latency per reply rather than per call?
Yes — turn_timing is posted per reply, and the deterministic findings in the call review point at the specific slow replies rather than an average. An average hides the one 9-second gap that lost the caller.