Connect by JBRH Open Connect

The realtime voice engine

On the realtime engine Connect holds the call itself: audio goes to a model that listens and speaks at once, rather than to a carrier that transcribes a sentence and reads a reply back. A workspace turns it on by setting voice.engine = "realtime" and linking the number at the carrier to an inbound trunk. The worker carries no business logic — everything it knows arrives in one brief.

Status
Available What this means
Audience
both
Channels
phone
In the app
#/calls, #/phone-advanced
Last verified
Product version
6.3.2

What happens to a call on this path#

  1. TRIGGER — a call arrives on a number whose workspace has voice.engine = "realtime"
  2. USER / EXTERNAL EVENT — the carrier opens a SIP leg into the project's inbound trunk
  3. AUTH / WORKSPACE RESOLUTION — the number resolves to one workspace, and the handler enters it before reading anything
  4. INGEST / REQUEST — a dispatch rule spawns the connect-voice worker on the room; admission refuses not_ready if no worker has checked in
  5. CANONICAL RECORD — a Call row exists from acceptance, with the resolved voice settings snapshotted onto quality
  6. CLASSIFICATION / RESEARCH / REASONING — a realtime model listens and speaks on the audio stream; the greeting is played from audio synthesised during the ring
  7. KNOWLEDGE + MEMORY + RULES — voice_engine.brief() renders persona, line purpose, contact facts, standing instructions, knowledge and the hard rules once per call
  8. AUTONOMY / APPROVAL — outbound goes through comms_providers.dispatch_call with a placer, so consent, blocks, autonomy and budget gate it exactly as a carrier call
  9. ACTION / PROVIDER — the worker speaks; every sentence is posted to /api/voice/engine/spoken and the guardrails are re-checked after the fact
  10. RESULT — /engine/end runs the same voice.end_call the turn-based path runs
  11. RELATIONSHIP / TIMELINE / MEMORY — summary, lead, memory and follow-up run in the deferred digest
  12. AUDIT / USAGE / NEEDS YOU — model usage lands in costs; line health escalates into Needs You and drains as its cause clears

The stages that differ from the carrier path are the middle ones. Everything from the canonical record onwards is shared code, which is why changing engines does not change what a call leaves behind.

Turning it on#

  1. Set voice.engine = "realtime" on the workspace.

    Result Calls on the workspace's lines are held by the engine rather than by the carrier's turn loop.

  2. At the carrier, link the number to an inbound trunk. This is console work and cannot be done through an API on the live provider.

    Result Calls arrive as a SIP leg rather than as a series of webhooks.

  3. Set the trunk's origination URI to the LiveKit project's SIP host as host:5060, with no sip: prefix.

    Result The leg reaches the project. The host is the project-id subdomain — LIVEKIT_SIP_HOST — not the hostname in the server URL used elsewhere.

  4. Confirm a worker is checked in before testing.

    Result Admission refuses before a row exists when nothing has heartbeated for 90 seconds, so a test against an empty fleet tells you nothing about the configuration.

What the worker knows, and what it must not#

It knows the brief
Persona, the line's purpose, the contact's facts and standing instructions, knowledge, the hard rules and the supervisor's guidance — rendered once per call by voice_engine.brief().
It does not know the business
No queries, no rules engine, no records. If a fact is not in the brief, the voice does not have it, and that is deliberate: the worker is the mouth, not the mind.
It reports rather than decides
Sentences to /engine/spoken, events to /engine/event, the ending to /engine/end. Guardrails are applied on Connect's side, after the fact, where the authority lives.
It cannot open its own door
The worker's endpoint is authenticated by MAYA_VOICE_ENGINE_TOKEN over loopback. With no token set it answers 503 — closed, never open.

Keeping the brief the only channel has a cost worth naming: prompt size is latency. Instructions of 7,500 characters measured a 1.2–1.8 s first token; 9,600 characters measured 2.3–3.4 s. That is why the budgets are fixed — four facts and 2,000 characters of knowledge, 700 for memory, 600 for the contact block — rather than left to grow.

What this engine buys, and what it costs#

It buys interruption and speed. A caller can speak over the voice and be heard, a one-word answer starts a turn, and the default model reaches first audio in about 0.6 s on the host — against 1.2–2.2 s for the 2.5-generation native-audio model.

It costs money and flexibility. A live call is billed on audio tokens — four times the same model's text rate on input and roughly 2.7 times on output — and a Live session is re-billed for its whole context every turn, with production calls running 92% carried context by input token. The default model's session is also immutable, so mid-call notes and supervisor guidance cannot reach it: the worker records the attempt as undelivered rather than pretending it landed. Choosing the 2.5 model buys mid-call steering back at roughly a second per reply.

Which AI voice answers: Connect Live Voice or Connect Indic Voice#

Two AI voices can hold a call on this path, and each calling channel has its own setting: phone calls and WhatsApp calls are decided separately. Connect Live Voice is one model that hears, decides and speaks, which gives the fastest replies. Connect Indic Voice is a chain built for Indian languages: one service turns the caller's sentence into words, a language model answers, and a speech service says the answer, at the lowest cost per minute. Because each reply waits for the caller's sentence to end and for its words to be recognised, it begins later than a Connect Live Voice reply.

Connect allows on the channelWhat your calls there useWho decides
Both voicesYour admin's choice; until one is made, Connect's default for your accountA workspace admin, in Calling → Call Receptionist → AI Voice Engine
One voiceThat voice, automaticallyConnect: there is nothing to choose
NeitherNo AI voice. An inbound caller hears the line's closed message, and an outbound call is refused before anything ringsConnect: the panel says the AI voice is off on that channel

Which voices an account may use is decided by Connect, for every account and for one account, never inside the workspace. A team member who is not an admin sees the same panel read-only and is told who can change it. A change is used from the next call: a call already in progress keeps the voice it started with, because the voice is fixed when the call is briefed.

Connect never moves a call to a voice the account may not use. If Connect Indic Voice cannot answer, because its service refuses the readiness check made before the call, the call falls back to Connect Live Voice only where that voice is allowed on the channel; where it is not, the call is refused with the closed message rather than switched. If Connect cannot read which voices are allowed at all, it refuses the call rather than guess.

GET /api/v1/voice/engines

PUT /api/v1/voice/engines/whatsapp
Content-Type: application/json

{"engine": "indic"}

The same panel is open to the Calling API. GET /api/v1/voice/engines returns, per channel, the voice the next call uses, the voices allowed there, where the answer came from and whether the caller of the API may change it. PUT with "live", "indic" or "" (back to Connect's default) changes one channel. A team member is refused with admin_only, a voice the channel does not allow with engine_not_allowed, a channel with only one voice with nothing_to_choose, and any other value with unknown_engine.

Questions#

Do I have to move to the realtime engine to use Connect on the phone?

No. The turn-based carrier path is the default and needs no trunk, no worker and no fleet to run. The realtime engine is what you choose when being interruptible matters more than simplicity.

Can a colleague steer a live call on this engine?

Guidance is delivered through the supervisor panel and never reaches the caller — it goes into the system prompt and nowhere else. Whether it reaches the model at all depends on the model: the default 3.1 session is immutable and the attempt is recorded as guidance_undeliverable, while the 2.5 model accepts it.

Can phone calls and WhatsApp calls use different AI voices?

Yes. Each channel has its own setting, within the voices Connect allows on it, so a business can answer phone calls with Connect Indic Voice and keep WhatsApp calls on Connect Live Voice, or the other way round.

Why can I not choose the AI voice?

Either only one voice is available to your account on that channel, so there is nothing to choose, or you are not a workspace admin: team members see the panel read-only. Which voices an account may use is set by Connect, not in the workspace.

What happens if the model refuses to start a session?

It is retried once on the same model and once on the mutable fallback before the call is given up, and model_session_restarted goes on the record. A model id that is a Vertex path or otherwise malformed is refused up front as bad_model, before a row exists.