Connect AI realtime speech model
Connect AI's live phone voice runs on a third-party realtime speech model reached over an API. One streaming session listens and speaks at once, with no separate speech-to-text and text-to-speech steps. The same service transcribes WhatsApp voice notes and voices the office's spoken replies when a workspace turns that on. Every use is metered, guarded by a circuit breaker and checked by Connect's own rules.
What it is#
A realtime speech model takes audio in and produces audio out within one streaming session. It understands the caller and answers in a single step, which is what makes natural interruption (barge-in) and low latency possible. See speech-to-speech models for the general idea.
In Connect it sits behind Connect AI. The provider and the exact model are settings that the platform operator chooses and can change. Customers see Connect AI, and the Owner's console keeps the full technical detail. This page describes how the model is used and governed, not whose it is.
- Category
- Model API (realtime audio, transcription, speech)
- Reached through
- The LiveKit Agents provider plugin on calls (pinned at 1.8.0); HTTPS elsewhere
- Configured
- Keys in the host's protected configuration or the workspace's encrypted settings, never in code
- Customer-facing name
- Connect AI
Where Connect uses it#
| Place in Connect | What it does there |
|---|---|
backend/voice_worker/agent.py | The realtime session on a call, through the LiveKit provider plugin |
| Voice-note transcription | WhatsApp voice notes into text, metered per note |
office_voice.py | Spoken replies from Connect's Employees, only when the workspace's own voice is on |
ai_brand.py | Any provider error a customer would see becomes a plain Connect sentence |
How it connects#
- A feature asks for a model session with its brief and limits
- The shared circuit breaker refuses fast if the provider is failing
- The request goes out with the configured model and key
- Usage comes back and is written to costs by modality (audio and text priced separately)
- Connect's rules check the result before anything is said or sent
Benefits and limits#
What it gives Connect:
- Native audio means low latency and natural interruption on calls.
- Costs are recorded per call and per modality, so spend is visible rather than estimated.
- Because the provider is a setting behind Connect AI, it can change without changing what a customer is told.
Where it stops, and what Connect does about it:
- The voice settings are locked (the call-quality lock). A change of model or engine option needs the owner's approval and a measured call.
- A model's answer is checked, not trusted. Guardrails are re-applied to every spoken sentence after the fact (unsupported claims).
- Rate limits and outages happen. The breaker opens and the call or task fails honestly instead of hanging, and an unlisted model is priced at the dearest listed rate, never at zero.
Security and privacy#
- Keys live in protected host configuration or encrypted workspace settings.
- Only what a feature needs goes into the brief (system prompts and instructions).
- The processors that personal data reaches are named in the privacy policy at
/privacy/, as the law requires.
Licence, cost and versions#
| Question | Answer |
|---|---|
| Licence | Commercial API under the provider's terms |
| Cost | Per token or per second of audio. Connect records each call's actual usage and cost by modality |
| Version | The model is a setting; the LiveKit provider plugin is pinned at 1.8.0 |
| Compatibility | Some engine options apply only to native-audio models. The voice brief sends the options the configured model understands |
Troubleshooting#
- Calls fail at start with an engine error
- The provider refused the session, the key is invalid, or quota is exhausted. The call's engine events name which, in the Owner's console.
- Voice notes are not transcribed
- The breaker is open after repeated provider errors, or no key is configured.
- Costs look higher than expected
- Audio is priced differently from text. Check the per-modality split (what a model call costs).
Official references#
Questions#
Which company's model answers my calls?
Connect AI is the product you use. The model underneath is a supplier choice the platform operator makes and can change. The privacy policy names the processors your data reaches.
Can a caller tell they are speaking with software?
Connect's automated Employees are never presented as people, and the disclosures a business must give stay in place.