# Reply latency on a call

Latency is timed on the wire: from the moment the caller's speech stops to the first audio coming back. The target is 2.5 seconds and 4 seconds is marked slow, but the realtime path has a floor — the model's own first token plus its end-of-turn detection — and the best measured call sat at a 3.3-second median. When the model is most of the wait, the review says so and offers no dial to turn.

- **Status:** Available
- **Audience:** both
- **Channels:** phone
- **In the app:** #/calls
- **Last verified:** 2026-09-10
- **Canonical:** https://connectbyjbrh.com/docs/phone/latency/

## Measured on the wire, not from the transcript

The worker's `TurnClock` starts when the caller's user state leaves `speaking` and stops when the agent state enters `speaking` — last word in, first audio out. Every reply posts a `turn_timing` event with that number.

The obvious cheaper approach is to subtract the timestamps on two transcript rows, and it is wrong in a way that flatters the system enormously. The first real call measured that way showed **"8 ms"** beside replies the caller had waited seconds for, because a transcript row is written when a sentence is *complete*, not when it started. Any latency figure taken from transcript rows is measuring transcription, not conversation.

> **Note** This is the reason a call review and a stopwatch agree. If they ever disagree on a Connect call, the stopwatch is right and something is wrong with the events, not with the perception.

## The floor, and where it comes from

| Model or condition | First token | Note |
|---|---|---|
| Gemini 3.1 Flash Live | 0.6 s | The default model, measured on the host |
| 2.5 native audio | 1.2–2.2 s | Chosen when mid-call notes matter more than a second per reply |
| 7,500-character instructions | 1.2–1.8 s | Prompt size is on the record as `session.instructions_chars` |
| 9,600-character instructions | 2.3–3.4 s | The same model; 2,100 more characters |
| Best measured call, median reply | 3.3 s | First token plus end-of-turn detection, end to end |

Two things fall out of that table. The first is that the realtime floor is not one to two seconds and nobody should promise it: end of turn has to be *detected* before generation can start, and the detection is not free. The second is that the prompt is a latency control — the gap between 7.5k and 9.6k characters is a second and a half of a caller's life, per reply.

That is why the [call brief](/docs/phone/call-context/) is built against character budgets rather than assembled from whatever is available: knowledge at 4 facts and 2,000 characters, memory 700, the contact block 600, a human voice profile under about 2,500.

## A slow reply against a slow model

`voice_quality.review` judges against `TARGET_RESPONSE_MS` (2.5 s) and `SLOW_RESPONSE_MS` (4 s), and then does something less usual: it asks how much of the wait was the model's own first token. Past `MODEL_BOUND_SHARE` — **60%** — the finding says the model is the cause and offers no setting, because there is no setting.

Those `MODEL_LIMITED` findings are listed and counted but do not lower `score`; `score_all` is the old number that included them. A tuning score that drops for something no control can change teaches the person tuning it to ignore the score.

1. Check whether the finding is marked model-limited.
   - Result: If it is, no amount of Voice Lab work will move it. The remaining levers are the model choice and the prompt size.
2. If it is not, look at the reply length and the instruction size before the exotic settings.
   - Result: Shorter replies and a smaller brief move first-token time measurably; most other knobs do not.
3. Re-run and compare `turn_timing` on a fresh call, not the same one.
   - Result: The resolved settings are snapshotted on `call.quality`, so each review judges the configuration that actually ran on that call.

## What did not cost anything

The 2026-09-06 change that shortened replies is the useful counter-example. Asking for one thought, one question and under 25 words moved the median words per reply from 23–40 to 19 and removed stacked questions entirely, at a reply median of 2.9 s and **no latency regression**. Speaking less is not slower; it is very often how a call starts feeling fast.

## Questions

### Why is my call slower than the 0.6-second first-token figure?

Because first token is not a reply. A caller waits for end-of-turn detection, then the model's first token, then enough audio to be audible. The 3.3-second median on the best measured call is the honest end-to-end number for the realtime path.

### Does a longer knowledge base slow every call down?

Only the part of it that reaches the prompt, which is capped: four facts and 2,000 characters. That cap exists because the measured difference between a 7.5k and a 9.6k prompt was more than a second on first token, and knowledge is the easiest thing to let grow without noticing.

### Can I see latency per reply rather than per call?

Yes — `turn_timing` is posted per reply, and the deterministic findings in the call review point at the specific slow replies rather than an average. An average hides the one 9-second gap that lost the caller.

## Related

- [The response watchdog](https://connectbyjbrh.com/docs/phone/response-watchdog/)
- [Turn detection and end of speech](https://connectbyjbrh.com/docs/phone/turn-detection/)
- [The call brief](https://connectbyjbrh.com/docs/phone/call-context/)
- [Call quality review](https://connectbyjbrh.com/docs/phone/voice-lab-review/)
- [What every character of a prompt costs on a live call](https://connectbyjbrh.com/research/prompt-size-and-first-token/)

## What this page is based on

- Connect phone source pack — `docs-source/sources/PHONE.md` §6, latency on the wire
- Connect phone source pack — `docs-source/sources/PHONE.md` §7 and §12, first tokens and the floor
- Connect capability registry — `docs-source/facts.py` (`MEASURED`)