System prompts and instructions
A system prompt is the standing instruction a model receives before the conversation: who it is acting as, what it may and may not do, and how to write. It is re-sent on every single call, so its length is not a style choice — on the voice path it is the largest thing standing between a caller finishing their sentence and hearing a reply.
What belongs in it, and what does not#
| Content | Where it belongs | Why |
|---|---|---|
| Role, tone, register | Instructions | Constant across every call; genuinely standing |
| Hard prohibitions | Instructions, and a withheld capability | Say it and remove the tool; the sentence alone is a request |
| Output shape | Instructions | Cheap to state, expensive to correct afterwards |
| Business facts and policies | Knowledge | Facts change; retrieval means editing one record instead of a prompt |
| What is known about a person | Memory, contact tier | Only some calls need it, and it must be inspectable and deletable |
| Worked examples | Instructions, sparingly | Effective and heavy. Two good ones beat eight |
The common mistake is putting the fourth row in the first. It works, briefly, and then the price list changes and nobody can find where the old one was written. Facts in a prompt cannot be audited, corrected by a person, scoped to one channel, or deleted on request.
Layering and precedence#
Connect assembles what a model receives from several layers, and the order is part of the design: the product's own operating rules, then the workspace's behaviour and Standing Instructions, then the voice or channel style, then the grounding and memory blocks for this specific piece of work.
- Narrower does not mean unlimited. A workspace instruction shapes tone and priorities; it does not lift a product-level prohibition.
- Contradiction is the enemy of adherence. Two layers disagreeing produces inconsistent behaviour, not a compromise, and the inconsistency is hard to attribute later.
- Every layer costs on every call. Adding a paragraph to the workspace layer is a per-call charge, not a one-off.
- Instructions are not a memory. They are the same on every call by definition, so anything that changes belongs in a record.
Does Connect use system prompts?#
Used, and treated as a budget with measurements behind it. On the realtime voice path an instruction block of around 7,500 characters produced a first token in 1.2–1.8 seconds; at around 9,600 characters the same model took 2.3–3.4 seconds. That is not a smooth curve you can trade against — it is the difference between a reply and a pause the caller talks into. A human voice profile's prompt block is held to about 2,500 characters for the same reason.
Two numbers give the target its shape. A reply is aimed at 2.5 seconds and treated as slow past 4 seconds, and about 60% of the remaining latency on the best-configured calls was model-bound rather than anything a setting could move — see findings a setting cannot fix and what every character of a prompt costs on a live call.
Behaviour and Standing Instructions are edited from Connect Rules, which is the screen that holds behaviour, rules and the intelligence boundary. Keeping them on a screen rather than in a deployment means a change is visible, attributable and reversible — which matters more than it sounds, because an instruction change alters every future answer at once.
Symptoms of an overloaded instruction block#
- Late first token on voice
- The most direct symptom, and the easiest to measure. Time to first token, not total reply time.
- Selective obedience
- Long instruction sets are followed unevenly; the rule in the middle is the one that slips.
- Contradictions surfacing as randomness
- The same question answered two ways on different days usually means two layers disagree.
- Drift after an edit
- A paragraph added to fix one case changes three others, because nothing in a prompt has a scope.
- Nobody can say why it says that
- The block has become a sediment of fixes with no owner per line.
Questions#
Why not put our whole policy handbook in the instructions?
Because you would pay for it on every call, wait for it on every first token, and lose the ability to correct one clause without editing a prompt. Handbook content belongs in Knowledge, where it is retrieved when relevant and edited by whoever owns it.
Does a longer instruction block make the model more obedient?
Past a point it does the opposite. Adherence degrades as instructions lengthen, and the rules least likely to be followed are the ones buried in the middle. Fewer, sharper rules beat more of them, and a withheld capability beats both.
Are the same instructions used on email and on a call?
No. The layers differ by channel because the constraints differ: a call is latency-bound and spoken, mail is not. That is why the voice budgets are tight and the voice style layer exists separately.