# LiveKit Agents and the voice worker

LiveKit is the realtime media platform under Connect's live phone and WhatsApp calls. A call from the carrier arrives over SIP into a LiveKit room. A dispatch rule starts Connect's `connect-voice` worker in that room, and the worker, built on the LiveKit Agents framework, connects the caller's audio to a realtime speech-model session. The worker knows no business logic. It asks the application for everything.

- **Status:** Reference
- **Audience:** both, developer
- **Last verified:** 2026-11-02
- **Canonical:** https://connectbyjbrh.com/docs/technology/livekit/

## What it is

LiveKit is an open-source WebRTC platform from LiveKit, Inc. It has a media server (an SFU), client SDKs, a SIP service that bridges phone calls into rooms, and the Agents framework for writing programs that join a room as a participant. LiveKit Cloud is the company's hosted service.

The Agents framework handles the hard real-time parts: audio frames, voice activity, interruptions, and plugins for speech and model providers. Connect uses a model-provider plugin for the realtime speech model, the Silero plugin for host-side voice activity detection, and the noise-cancellation plugin.

**Category** — Realtime media and voice-agent framework
**Versions in Connect** — `livekit==1.1.17`, `livekit-api==1.2.1`, `livekit-agents==1.8.0`, the model-provider plugin at 1.8.0, `livekit-plugins-silero==1.8.0`, `livekit-plugins-noise-cancellation==0.3.0`
**Project** — A LiveKit project configured on the host (`LIVEKIT_URL` and its key pair)
**Official site** — [docs.livekit.io](https://docs.livekit.io/)

## Where Connect uses it

| Place in Connect | What it does there |
|---|---|
| `backend/voice_worker/agent.py` | The worker: joins the room, runs the realtime session, posts each sentence and event to the application |
| `backend/app/voice_engine.py` | Renders the call brief once per call and receives the worker's posts |
| `backend/app/whatsapp_calling.py` | WhatsApp calls through the same engine, with an output gate on that leg only |
| `backend/voice_worker/audio_gate.py` | The noise gate on the agent's voice for WhatsApp legs |

## How it connects

1. The carrier sends the call to the project's SIP host
2. LiveKit SIP creates a room, and a dispatch rule starts the worker
3. The worker fetches the brief from the application on the loopback port
4. Caller audio passes noise filtering, then the realtime model session
5. At the end the worker posts `/engine/end`, and the application writes the summary, memory and follow-up

## Benefits and limits

What it gives Connect:

- Barge-in, automatic language and one consistent voice, because the model listens and speaks at once.
- A thin worker: business rules stay in the application, where they are tested.
- Exact pins, proven together on a real call, so an upgrade is a deliberate, measured change.

Where it stops, and what Connect does about it:

- The calling quality is locked. Engine settings, filters, turn-taking and the tool set are pinned by a suite, and changing them needs the owner's approval and a measured call.
- The enhanced noise-cancellation plugin requires LiveKit Cloud.
- Realtime sessions are billed by audio duration on both LiveKit and the model provider. A worker never holds a session past its maximum duration.

## Security and privacy

- The worker talks to the application over loopback with its own token. The endpoint answers 503 when the token is unset, never open.
- LiveKit credentials live in the host's environment file.
- Recordings and transcripts follow the workspace's settings. A transcript is evidence, not a clock ([call transcripts](/docs/technology/call-transcript/)).

## Licence, cost and versions

| Question | Answer |
|---|---|
| Licence | LiveKit server, SDKs and Agents: Apache-2.0. The hosted service and its enhanced noise-cancellation models are commercial |
| Cost | LiveKit Cloud usage per its [pricing page](https://livekit.io/pricing). The model's audio is billed separately and metered by Connect per call |
| Version | Exact pins in `requirements-voice.txt` |
| Compatibility | Installed into the same venv as the application on the host; the web application never imports the realtime SDK |

## Troubleshooting

**The call connects but nobody speaks** — The worker was not dispatched or failed to start its model session. Check `journalctl -u connect-voice` and the call's engine events.
**The caller hears the agent's own echo or hiss on WhatsApp** — The WhatsApp output gate. It is locked: report it with the call, do not tune it ad hoc.
**Calls end at a fixed length** — The session reached `max_seconds`. That is the cost guard.

## Official references

- [LiveKit Agents](https://docs.livekit.io/agents/)
- [LiveKit SIP](https://docs.livekit.io/sip/)
- [Enhanced noise cancellation](https://docs.livekit.io/cloud/noise-cancellation/)
- [livekit/agents on GitHub](https://github.com/livekit/agents)

## Questions

### Does every call use LiveKit?

Calls on lines set to the realtime engine do. A line can also use the carrier's turn-based path, where the carrier plays Connect's written reply. Both end in the same call record.

### Why is the voice worker's SDK not in the main requirements?

So a broken audio stack cannot stop the web application, and the mail sync, from starting. The two run as separate services from one venv.

## Related

- [Realtime voice agents](https://connectbyjbrh.com/docs/technology/realtime-voice/)
- [Connect AI realtime speech model](https://connectbyjbrh.com/docs/technology/realtime-speech-model/)
- [Vobiz telephony carrier](https://connectbyjbrh.com/docs/technology/vobiz/)
- [Voice activity detection](https://connectbyjbrh.com/docs/technology/vad/)
- [SIP trunks](https://connectbyjbrh.com/docs/technology/sip-trunk/)

## What this page is based on

- `requirements-voice.txt`
- `docs/agents/9a-phone-system.md`
- `backend/voice_worker/agent.py`
- `ops/connect-voice.service`
