Structured business memory instead of a longer prompt
A longer prompt is not a memory. It is the same information re-sent on every turn, paid for in seconds before each reply and in a context re-billed each time, and it ends when the session does. A tiered store keeps what is true at the level it applies to, resolves only what the work in hand needs, and can be corrected by the person who noticed it was wrong.
What a context window is, precisely#
It is a window, it belongs to one session, and its contents are presented again on every turn. On a live call that is not a metaphor: a session is re-billed for its entire context every turn, and production calls measured here run at 92% carried context by input token. The caller's new sentence is a rounding error beside the material being resent.
Because it is per session, nothing in it survives the call. The next conversation with the same person starts with whatever was assembled for it — which means the question was never *how big is the window*, it was always *what gets assembled*.
The measured price of putting it in the prompt#
| Instructions | First token |
|---|---|
| 7,500 characters | 1.2–1.8 s |
| 9,600 characters | 2.3–3.4 s |
Roughly a quarter more text, roughly double the wait, in front of every reply. Only one lever measured here moves further: the model itself, at 0.6 s to first audio on Gemini 3.1 Flash Live against 1.2–2.2 s on 2.5 native audio. Everything a business might want to *tell* the agent competes inside the smaller of those two levers.
The cost is also charged in the expensive modality. A live call is billed on audio tokens at roughly 4× the same model's text rate on input and about 2.7× on output, so a paragraph nobody needed is paid for once per turn, at the higher rate, for the length of the call.
What a four-tier store gives instead#
| Property | Longer prompt | Tiered store |
|---|---|---|
| Scope | One flat blob for every situation | workspace → channel → endpoint → contact, narrowest wins |
| Lifetime | The session | Durable; survives the call, the process and the deploy |
| Correction | Edit the template for everybody | Edit or forget one row, at the tier that owns it |
| Attribution | Unknown once assembled | The tier a direction came from is visible in the viewer |
| Cost at use | Re-sent every turn | Resolved once per piece of work, inside a character budget |
| Absence | Invisible | An empty tier is returned, so *nothing is set here* is an answer |
The tier hierarchy is deliberately the same one autonomy uses, so a person who understands why a rule for one contact beats a rule for the channel already understands why a memory for one contact does too. Two hierarchies would be two mental models for one idea.
Selection is the work, and it has a cost of its own#
voice_engine.brief() renders the persona, the line's purpose, the contact's facts and standing instructions, knowledge, the hard rules and the supervisor's guidance once per call — inside budgets of 4 facts or 2,000 characters of knowledge, 700 characters of memory and 600 for the contact block. Those numbers are what turn *what should the agent know* from a wish into a decision.
The honest cost is that selection can be wrong, and an omission is not automatically visible: an agent that was never given the relevant fact behaves like an agent that ignored it. That is the argument for keeping the store readable at every tier rather than for making the selection cleverer — a person looking at the tiers can see what was available to be chosen.
Where a bigger window is genuinely the answer#
- Within one long document or one long call, more room is more room. The argument here is about facts that outlive the session, not about working material inside it.
- Where nothing needs correcting. A stable instruction set with no per-customer variation has little to gain from tiers.
- Not for grounding. Knowledge is the store an answer must be grounded in; memory shapes handling. Putting commercial facts into memory because it is closer to hand is a common and expensive substitution.
- Recall quality is unmeasured here. Whether the selected memory was the right memory is UNKNOWN as a rate, and would need a review of calls where the answer was wrong rather than slow.
Questions#
If the model has a very large context window, why not use it?
Because the window is refilled and re-charged every turn, and the wait it adds lands in front of each reply rather than once at the start. Size also does not give you the properties that matter operationally — scoping, correction and durability — which are properties of a store rather than of a prompt.
Does this apply outside voice?
The latency argument is sharpest on a live call, where a second of assembly is a second of silence. The durability and correction arguments apply everywhere: an email agent given the same fact by prompt has no way for a person to change it for one customer.
How large should the budgets be?
Large enough that the agent is not ignorant, small enough that the first reply arrives. The measured pair — 7,500 and 9,600 characters — is the evidence there is here, and it is why the per-block budgets exist as numbers rather than as advice.