Turning on the realtime engine, end to end
Setting voice.engine = "realtime" is the smallest part. The call has to arrive somewhere else entirely: the number is linked at the carrier to an inbound trunk pointed at the LiveKit project's SIP host, a dispatch rule spawns the connect-voice worker on the room, and the worker must be running, warmed and authenticated before the first caller. Three of those steps are console-only.
The chain, when the engine changes#
- Trigger — a workspace decides it wants the voice to listen and speak at once.
- User event — an administrator with carrier console access and workspace access begins.
- Authentication and workspace resolution — the engine setting is per workspace; the trunk is per number at the carrier.
- Ingest — the number is linked to an inbound trunk whose origination URI is the LiveKit project's SIP host.
- Canonical record —
voice.engineon the workspace, alongside the unchangedchannel_routesrow. - Reasoning — the model is chosen; the default reaches first audio in about 0.6 s.
- Knowledge, memory and rules —
voice_engine.brief()renders persona, purpose, contact facts, knowledge and hard rules once per call, under fixed character budgets. - Autonomy and approval — nothing about consent, budget or autonomy changes; both engines pass the same gates.
- Action through a provider — a dispatch rule spawns the
connect-voiceworker on the room, and the model speaks into it. - Result — a real inbound call answered by the worker, with
greeting.prewarmedon the event. - Relationship, timeline and memory —
/engine/endruns the samevoice.end_callthe carrier path runs. - Audit, usage and Needs You — model usage is written into
costs, andline_healthstarts reporting on the worker.
The three things no API will do for you#
These are the steps that consume an afternoon if they are wrong, because each fails silently: registration succeeds, configuration looks complete, and calls arrive nowhere.
- Linking the number to the trunk happens in the carrier's console. There is no API call for it. A number that is not linked routes to the old application, or to nothing.
- The origination URI is
host:5060with nosip:prefix. The scheme that looks obviously correct is rejected. - The SIP host is the LiveKit project's own subdomain (
LIVEKIT_SIP_HOST, the project-id subdomain) — not the host in the server URL you use for everything else.
The worker, and what it needs to be ready#
| Stage | What you see | What changes | What can fail |
|---|---|---|---|
| Worker running | The Phone screen reports a live worker | Heartbeats carry one-minute load per core and the calls held | No check-in for 90 s and ring refuses not_ready — retryable, so a drain picks it up |
| Door authenticated | Nothing when correct | The worker posts to a loopback endpoint | Without its token the door answers 503; it is never open |
| Greeting warmed | First greeting is instant | Each idle process synthesises the greetings into its own cache | Every process asking at once once earned a 429 and gave the greeting up for that process's life |
| Capacity | Calls are accepted | Admission control and the worker agree on the same busy threshold, 0.85 | Every live worker over the threshold — the call is refused rather than queued into silence |
| Deploys | Calls in progress finish | A drain window of 180 s, with the service allowing 210 s | A restart during a call, if the drain is shortened below the service's own stop timeout |
Warm-up now spreads processes over a stagger interval and retries a refused synthesis with backoff when the refusal is one worth retrying — a rate limit or a server error, never a bad key. It runs inside process initialisation, whose default bound was shorter than the warm-up's own budget, so the worker asks for a longer initialisation timeout and keeps its warm-up budget inside it.
What changes for the caller, and what does not#
What changes: the voice can be interrupted mid-sentence and will stop; the greeting is spoken the instant the caller picks up because it was synthesised during the ring; replies are paced by the model rather than by the carrier's speech detection. The realistic floor is the model's first token plus its end-of-turn detection — 3.3 s median reply on the best measured call.
What does not change: the same brief, the same rules, the same consent and budget gates, the same voice.end_call, the same cost ledger. The worker knows no business logic at all. It is handed a brief and posts every sentence back to be re-checked against the guardrails after the fact, which is the only honest arrangement when the model has already spoken.
Questions#
Do both engines need separate carrier setup?
The turn-based path needs the number pointed at the voice application. The realtime path needs the number linked to an inbound trunk instead. They are different destinations for the same number, which is why switching engines is carrier work and not only a setting.
Which model should a line use?
The default is the faster of the two, at roughly 0.6 s to first audio against 1.2–2.2 s. Its session is immutable, so mid-call notes and a supervisor's guidance cannot reach it and are recorded as undelivered. Choose the older model when steering a live call matters more than a second per reply.
What happens on deploy while a call is in progress?
The drain window lets calls in progress finish, and the service allows longer than the drain before it kills the process. Calls that cannot be accepted during the drain are refused in a retryable way rather than dropped.