What every character of a prompt costs on a live call
Instruction size is recorded on every realtime session, and the two measured points are far apart: 7,500 characters of instructions gave a 1.2–1.8 s first token, 9,600 gave 2.3–3.4 s. On a call, a second of prompt is a second of silence in front of every reply. The budgets that followed cap knowledge, memory and the contact block by characters rather than by how useful each addition feels.
The measurement and its method#
session.instructions_chars is on the record for every realtime session, so prompt size is a measured property of a call rather than an estimate from the template. Timing is taken on the wire by TurnClock, not from transcript rows, which is what makes the two figures comparable at all.
Two sizes, two ranges. A 28% increase in instruction characters roughly doubled the time before the caller heard anything. That is not a throughput cost that averages out over a call; it is a delay in front of every single reply, and callers hear the first one loudest.
Why it is worse than the latency figure alone#
A Live session is re-billed for its entire context on every turn. Production calls measured here are 92% carried context by input token — the caller's new sentence is a rounding error next to the prompt being resent. A live call is also billed on audio tokens, which run about 4× the same model's text rate on input and roughly 2.7× on output, so the multiplier lands on the expensive modality.
So an extra paragraph of instructions is charged once per turn for the life of the call, and delays every one of those turns. The two costs point the same way, which is unusual and convenient: there is no trade to argue about.
The budgets that came out of it#
| Block | Budget | Why it is bounded there |
|---|---|---|
| Knowledge | 4 facts / 2,000 chars | Enough to ground an answer; past this the model is being given a manual to read aloud |
| Memory | 700 chars | What is true about this person, not their history |
| Contact block | 600 chars | Identity and standing instructions, not a profile |
| Human voice profile | under ~2,500 chars | A whole person's speech description still has to fit beside the rest |
voice_engine.brief() renders the persona, the line's purpose, the contact's facts and standing instructions, knowledge, the hard rules and the supervisor's guidance once per call. Everything in that render competes for the same seconds, which is why the budgets are per block: without them, the block that grows is whichever one somebody edited most recently.
Model choice moves the floor further than the prompt does#
Gemini 3.1 Flash Live measured 0.6 s to first audio on the host, against 1.2–2.2 s for 2.5 native audio. That single choice is worth more than any plausible prompt reduction — and it is not free: 3.1 refuses affective dialog (error 1007) and the SDK marks its session immutable, so mid-call notes and supervisor guidance cannot reach it. The worker records the stalled reply in silence (nudge {via: none}), notes as undelivered and guidance as guidance_undeliverable. 2.5 is the right pick when mid-call steering matters more than a second per reply.
Output length behaves differently from input length. Cutting the median reply from 23–40 words to 19 produced no latency regression — shorter answers were simply better answers. Shorter *instructions* buy time; shorter *replies* buy attention. They are not the same lever.
Limits of these numbers#
- Characters are a proxy for tokens, and the ratio differs by language and script.
- Both measurements were taken against one model family on one path; another provider's behaviour is UNKNOWN here.
- Network conditions and model-side load are not controlled for, which is part of why the results are given as ranges rather than as single values.
- The floor underneath all of it is the model's first token plus its end-of-turn detection — 3.3 s median on the best measured call. A prompt small enough to cost nothing still cannot beat that.
Questions#
Is it not simpler to give the model everything and let it choose?
That is what the budgets are refusing. Everything means the whole knowledge base and the whole relationship history resent on every turn, at audio-token rates, in front of every reply. Selection has a cost — the right fact can be left out — but it is a bounded cost, and it is visible in the review when it happens.
Which block should be cut first when the brief is too long?
Knowledge, because it is the block most likely to contain material the caller will never ask about, and because four grounded facts answer more questions than twenty ungrounded paragraphs. The contact block and memory are already small and are what make the call feel like a continuation rather than a first contact.
Does the same reasoning apply to email drafting?
The economics are different: a mail draft is one generation, not one per turn, and nobody is listening to silence while it runs. Context discipline still helps quality, but the latency argument on this page is specific to a live conversation.