Context compression on a live call
Context compression replaces the older part of a conversation with a shorter summary so the window stops growing. On a live voice call it is a cost control before it is a capacity one, because the whole context is charged again on every turn. Connect sizes the trigger explicitly on the realtime path; left at its default it is the model's entire window, which is the same as off.
The bill grows faster than the call does#
A conversational model is stateless between turns. Each turn re-sends everything the model needs to know — the system instructions, the knowledge, the whole dialogue so far — and each turn is charged for all of it. Turn twenty on a call pays for turns one to nineteen again.
On text that is annoying. On a live call it is the dominant cost, because audio input tokens run at roughly four times the same model's text rate and audio output at nearly three times. Measured on production calls here, 92% of input tokens were carried context rather than anything new the caller had just said.
Trigger and target are two different numbers#
A sliding-window compressor needs both, and confusing them produces either a compressor that never runs or one that runs on every turn.
- Trigger
- The context size at which compression happens. Below it nothing is touched. Set it too high and the call ends before it is ever reached.
- Target
- The size the compressor aims to leave behind. The distance between trigger and target is the headroom you buy — a target just under the trigger means compressing again almost immediately.
- The recent tail
- The last few turns are kept verbatim. Summarising what the caller said ten seconds ago is how a voice agent starts asking for a detail it has already been given.
- The summary
- One model-written block standing in for the compressed turns. It costs a model call to produce, so a compressor that fires constantly has a bill of its own.
The pairing matters more than either number. A trigger at 60% of the window with a target at 30% compresses a handful of times across a long call; a trigger at 90% with a target at 85% compresses continuously and spends more on summarising than it saves.
Does Connect use context compression?#
Used on the realtime voice engine, and deliberately sized rather than defaulted. The compressor is configured through brief.COMPRESSION_TRIGGER_TOKENS and brief.COMPRESSION_TARGET_TOKENS. Its default trigger is the model's whole context window, which means a configuration nobody has thought about does nothing at all: the call ends long before the window fills.
Compression also does structural work beyond cost. Together with session resumption it is what lets a long call survive the provider's session limit, because a conversation small enough to carry is a conversation that can be handed to a reconnected session.
The instruction block sent at the start of a call is separately budgeted, for a related reason: prompt size shows up as first-token latency. Measured on this stack, a 7,500-character instruction gave a 1.2–1.8 s first token and a 9,600-character one gave 2.3–3.4 s. Hence the fixed ceilings on what may go into it — four knowledge facts and 2,000 characters, 700 characters of memory, a 600-character contact block.
What you lose when you compress#
Watch which turns fall into the summary first.
Result The oldest ones — which on a phone call are often the ones that established who the caller is and what they rang about.
Check what the summary keeps about identifiers.
Result A phone number or an order reference read out early is exactly the kind of detail a prose summary rounds off. Anything a call must not lose belongs on the record, not in the context.
Compare the reply after a compression with the ones before it.
Result A change of register or a repeated question straight after the compressor fires is the visible symptom of a summary that dropped something the model was relying on.
Compression is lossy by construction and the loss is not evenly distributed: facts stated once and never repeated go first. That is a reason to write durable things down through the call's own tools — a follow-up, a lead, a memory — rather than trusting the window to still be holding them at minute eleven.
Questions#
Is compression the same as a shorter context window?
No. A smaller window is a hard limit that errors when you cross it. Compression keeps the conversation going by rewriting its older half, so the call continues at a lower carried cost. They interact, though: compression is what keeps a conversation inside whatever window the session actually has.
Does compressing a call make it faster?
Usually a little, because there are fewer input tokens to process before the first token comes back. It is not the main lever on latency here — on the realtime path the floor is the model's own first token plus its end-of-turn detection, measured at a 3.3 s median reply on the best call recorded.
Where do I see what a call actually cost?
The per-call voice profiler reports what was measured and says plainly what was not, rather than estimating the gap. It is part of the usage ledger, which is operator-only.