Retrieving Knowledge for one reply
Only a small, chosen part of your Knowledge reaches any single reply. The question is formed from the conversation rather than from its last line, material is selected by retrieval over the sources you supplied, and a budget caps what is carried — on a live call, roughly two thousand characters and about four facts. What was selected can be inspected before anything is sent.
The chain, stage by stage#
- Trigger — a reply is being written: an email draft, a WhatsApp answer, or a turn in a call that has to be answered in seconds.
- User or external event — the customer's message is already a canonical record; retrieval happens against a conversation, not against an inbox.
- Authentication and workspace resolution — the workspace decides which Knowledge exists at all. No source outside it is reachable, and there is no shared pool to filter afterwards.
- Request — the question is formed from the thread: what was asked, what has already been answered, and what the customer is actually trying to settle.
- Canonical record — the thread, its history and the person behind the address are loaded together, which is why the answer reads as a reply rather than as a response to one sentence.
- Reasoning — candidate material is drawn from the supplied sources and the extracted facts. There is no vector database and no external lookup: what you supplied is the whole search space.
- Knowledge, memory and rules — the budget is applied here. Knowledge, memory and the block describing who is being answered each get a share, and the shares are deliberately small.
- Autonomy and approval — grounding does not decide whether anything is sent. The channel's mode, at the narrowest applicable scope, decides that separately.
- Action — the reply is drafted against the selected material. A commercial term the selection does not support is refused rather than smoothed over.
- Result — a draft, held or sent according to autonomy. What it was grounded in is part of the record rather than something to be reconstructed.
- Relationship, timeline and memory — the exchange joins the person's timeline; anything durable learned in it belongs in memory rather than in the thread.
- Audit, usage and Needs You — the decision is recorded, the work is metered against the daily allowance, and anything requiring a person is ranked into Needs You.
The budget, and why it is small#
Everything carried into a reply competes for the same space, and on the phone that space is bought with time. Prompt size and first-token latency move together: measured instructions of about 7,500 characters produce a first token in roughly 1.2 to 1.8 seconds, and 9,600 characters push the same measurement to between 2.3 and 3.4 seconds. On a call, that difference is the gap between an answer and a silence.
| What is carried | Budget |
|---|---|
| Knowledge | About 2,000 characters, and at most four facts |
| Memory | About 700 characters |
| Who is being spoken to | About 600 characters |
Written channels are less brutal about latency, but the principle does not change: a reply grounded in four well-chosen statements is better than one grounded in forty pages of nearly-relevant material, and the second is not available at any price. Treat the budget as a design constraint on your sources rather than as a limit imposed on your answers.
How to write sources that survive selection#
- State the answer, do not surround it. A sentence that says what the delivery window is beats three paragraphs about how delivery works.
- One claim per fact. A fact that bundles a price, a lead time and an exception can only be selected or not selected as a whole.
- Use the customer's words. Sources written in internal vocabulary are selected less often for questions asked in ordinary language.
- Keep one version. Two documents that both answer the same question are the most common cause of a confident, wrong reply.
- Put the exception next to the rule. Selected material arrives without its neighbours, so a caveat three pages away is a caveat that never appears.
Seeing the selection before a customer does#
Ask the retrieval test the question you are worried about, in the words a customer would use.
Result You see the material that would be selected, without sending anything, which turns an argument about model behaviour into a question about your sources.
If the right document is present but the wrong passage is chosen, add a direct fact stating the answer.
Result A short statement competes far better for a small budget than one line inside a long file.
If two sources both answer, retire one.
Result Selection stops being a coin toss between two things your business would say differently.
For a phone answer, test the same question again after the change.
Result The call path carries the least of anything, so a source that only just works in email may not survive the tighter budget at all.
What retrieval cannot do is rescue a question that was never answerable. If your Knowledge does not carry the answer, the honest outcome is a refusal and an escalation to a person — not a plausible sentence assembled from adjacent material.
Questions#
Why does only a fraction of my Knowledge reach an answer?
Because everything carried competes for the same space, and on a call that space costs time before the first word is spoken. Grounding is selective by design: a few well-chosen statements answer better than a large amount of loosely related material, which mostly dilutes the answer.
Is the question just the customer's last message?
No. It is formed from the conversation — what was asked, what has already been answered, and what is still unresolved. That is why a reply to a two-word follow-up still makes sense, and why an ambiguous thread produces an answer to a reasonable reading rather than to a guess.
How do I see which sources an answer used?
Use the retrieval test before sending, and the decision record afterwards. Both show the selection rather than describing it, which is what you need when the disagreement is about whether the model or the source was at fault.
Does a bigger Knowledge base give better answers?
Not on its own, and past a point it does the opposite. More material means more candidates competing for the same small budget, so an unmaintained bank full of near-duplicates produces less reliable answers than a small, current, well-written one.