End-of-turn detection
End-of-turn detection decides whether a silence means 'your go' or 'I am thinking'. It is added to every reply before the model begins, which makes it the largest controllable term in how fast a voice agent feels. Connect's fastest setting waits 450 ms of silence, and a semantic path exists that varies the wait by what was said.
The timer, and the trade it forces#
The simple implementation waits for a fixed run of silence. Set it short and the agent interrupts anybody who pauses to think, spells a name, or reads a number off a screen. Set it long and every exchange gains that delay — not once, but on each of the twenty turns in a call.
There is no value that satisfies both. The distribution of human pauses inside an utterance overlaps the distribution of pauses at the end of one, and no threshold separates two overlapping distributions. This is why tuning the number feels like moving a problem around rather than solving it — because that is what it is.
Semantic endpointing#
The better approach judges completeness rather than duration. 'My number is nine eight —' is obviously unfinished; 'that's all, thanks' obviously is not. A model that reads the words and the prosody can shorten the wait when an utterance sounds complete and lengthen it when it does not, which is called dynamic endpointing.
Connect's host path combines a neural voice detector with an audio end-of-turn model, both running in the worker's own process, with dynamic endpointing, and every decision is recorded as an event so the behaviour can be reviewed after the call rather than argued about. It works only when the model's own detection is switched off — two components cannot both own the same decision.
One consequence of the host path is worth stating because it is counter-intuitive: on it, false-interruption resume is off. Resuming after a supposed false interruption needs a transcript to judge the interruption by, and on this path there is none at the moment the decision must be made — so every resume ended up talking over the caller. Switching the feature off was the fix.
Why it dominates the felt delay#
| Term | Order of magnitude | Controllable? |
|---|---|---|
| End-of-turn decision | Hundreds of milliseconds, on every turn | Yes — a session parameter, and the choice of detection path |
| Model first token | 0.6 s on the default model; 1.2–2.2 s on the older native-audio model | Only by choosing a model |
| Instruction size | 7,500 characters gave 1.2–1.8 s; 9,600 gave 2.3–3.4 s | Yes, and it is the cheapest large win available |
| Network and jitter buffering | Tens of milliseconds | Barely |
Two readings follow. First, halving the silence timer is a real improvement a caller notices, because it applies to every turn. Second, it is not the only large term — an over-long instruction block costs more than the timer does, which is why the budgets here are strict: four facts and 2,000 characters of knowledge, 700 for memory, 600 for the contact block.
The review that runs after each call knows the difference. When the model's own first token accounts for 60% of the wait, the finding says so and offers no setting to change, because there is not one.
Does Connect use end-of-turn detection?#
Yes, and which detector is authoritative is a per-workspace choice made in the Voice Lab: the model's own detection with a configurable silence threshold, or the host's semantic path with the model's detection disabled.
Above both sits a response watchdog, because end-of-turn detection can simply fail to produce a reply. If the caller stopped and nothing has started coming back after 5.5 s, the worker asks the model to generate. That default was originally 3 s, and it was worse: a nudge issued while a generation is already in flight cancels it, so an impatient watchdog produced first tokens of three to seven seconds on one measured call. Only speech disarms it.
Every decision is measured on the wire — the caller's last word to the first audio back — and never derived from transcript rows, which arrive when a sentence completes and would report a wait of milliseconds for a reply the caller waited seconds for.
Questions#
What silence threshold should I use?
Start at the fastest setting and listen for interruptions during numbers, spellings and addresses — those are where mid-utterance pauses cluster. If callers are being cut off there, the semantic path is a better answer than a longer timer, because a longer timer slows every other turn to fix a few.
Why did making the watchdog more aggressive make replies slower?
Because a nudge sent while the model is already generating cancels that generation and starts again. A watchdog is a recovery mechanism for a stalled turn, not an accelerator, and setting it below the model's normal response time guarantees it fires on healthy turns.
Can I make the agent never interrupt?
You can make it much less likely by lengthening the wait, at the cost of feeling sluggish on every exchange. What you cannot do is eliminate it, because deciding whether a pause has ended is a judgement made without the information that would settle it — the words the person has not said yet.