# Retrieving Knowledge for an answer For each question Connect narrows to active sources whose scope fits the work, drops any that fail their integrity hash or have unreviewed problems, scores every remaining passage, and returns the top few above a floor. The result is grouped into three labelled blocks so fact evidence and background are not read as the same kind of statement. On a call the cap is four entries and 2,000 characters. - **Status:** Available - **Audience:** both, developer - **Last verified:** 2026-09-10 - **Canonical:** https://connectbyjbrh.com/docs/knowledge/retrieval/ ## What is eligible before anything is scored 1. The source status is `active`. Nothing in `needs_review`, `disabled`, `error` or still processing is considered. 2. Its scopes include `all` or overlap the scopes the work asked for. 3. When the question needs fact authority, its authority is `verified_fact` or `owner_instruction` — everything else is excluded outright. 4. Its stored file still matches its SHA-256 hash. A mismatch drops it and moves it to `needs_review` on the spot. 5. It has no warnings or conflicts, or a person has reviewed them and that review still matches the current state of the record. The fifth condition has a subtlety worth knowing. A review is recorded against a fingerprint of the file hash plus the exact warnings and conflicts that were approved. Change the file, or have a new conflict appear, and the fingerprint no longer matches — so the old approval stops counting and the source goes quiet again. Approval is granted to a specific state, never to a title. ## How a passage is scored Lexical scoring is BM25 across the eligible passages, normalised so the best match for that question is 1. When embeddings are present the question is embedded too and the final score is 60% cosine similarity and 40% lexical; with no embeddings, the lexical score stands alone. Then three adjustments: | Condition | Effect | |---|---| | The whole question, at least eight characters, appears literally in the passage | +0.15 | | Authority is `owner_instruction` | +0.10 | | Authority is `verified_fact` | +0.07 | | Authority is `playbook` | +0.03 | | Authority is `roadmap` | −0.06 | Anything scoring 0.02 or below is discarded rather than padded in. What survives is sorted and cut to the limit — eight by default, thirty at the most. The authority adjustments are small on purpose: they break ties between passages that are already relevant, and are not large enough to drag an unrelated owner instruction into an answer about something else. ## What the model is handed The selected passages are not concatenated. They are sorted into three blocks, each with its own heading telling the model what that material may be used for: **Fact evidence** — `verified_fact` and `owner_instruction` passages. May be stated, but only as far as the text supports. **Owner playbook and instructions** — `owner_instruction` and `playbook` passages read as behaviour — how to handle this, not proof of a fact. **Reference, roadmap and unverified** — Context only. Explicitly marked *do not state as current fact*. Every passage arrives with a citation of the form `KB::`, its source title and its section. Those citations are what [auditing an answer](/docs/workflows/knowledge-audit/) later works from, and they are recorded against the reply, not just shown to the model. ## Sufficiency is a separate question from relevance Getting passages back does not mean the question can be answered. A separate check asks whether the retrieved fact-authority material is strong enough — by default a score of 0.36 or better — and reports whether that threshold was met, which titles met it and which citations support it. Commercial, legal and support commitments raise the bar further and require explicit owner authority rather than a topical match. That is why a reply can cite a relevant policy and still decline to give a number. See [claims Knowledge does not support](/docs/knowledge/unsupported-claims/) for what Connect does instead of guessing. ## Why the call budget is so small An email reply can afford eight passages. A realtime call cannot: the brief is sent before the first word and every character is paid for in latency — 7,500 characters measured 1.2–1.8 s to first token against 2.3–3.4 s at 9,600. So the call brief takes four entries and 2,000 characters of Knowledge, 700 of memory, 600 for the contact block and 500 for prior history, and the assembled size is recorded on the call so a slow first token can be explained rather than guessed at. Duplicated entries are the expensive failure here. Two Knowledge entries saying the same thing in different words occupy two of the four slots, and on a live call the whole brief is re-sent with the session on every turn. Near-identical entries are collapsed before the block is built, but collapsing is a repair, not a substitute for tidying the bank. ## Questions ### Why did a clearly relevant document not appear in an answer? Work down the eligibility list before suspecting the ranking. The usual answers are that the source is in `needs_review` after a warning nobody resolved, that its scopes do not include the kind of work in hand, or that the question needed fact authority and the source is only `reference`. ### Does a longer document rank higher? No. BM25 normalises for passage length, so padding a source does not buy it position. What does help is that the words a customer would actually use appear in the passage, and that the passage sits under a heading naming its subject. ### Can I see the scores? Yes — the search tool returns the same rows the reply path uses, with the score, the authority and the section on each. [Testing retrieval](/docs/knowledge/retrieval-test/) covers how to read them. ## Related - [How Knowledge is prepared for retrieval](https://connectbyjbrh.com/docs/knowledge/indexing/) - [Testing retrieval](https://connectbyjbrh.com/docs/knowledge/retrieval-test/) - [Claims Knowledge does not support](https://connectbyjbrh.com/docs/knowledge/unsupported-claims/) - [Producing a grounded answer, end to end](https://connectbyjbrh.com/docs/workflows/grounded-answer/) - [Reply latency on a call](https://connectbyjbrh.com/docs/phone/latency/) - [What every character of a prompt costs on a live call](https://connectbyjbrh.com/research/prompt-size-and-first-token/) ## What this page is based on - `backend/app/knowledge_bank.py` — eligibility, BM25 scoring, blending and the context blocks - `backend/app/knowledge.py` — coverage, critical topics and the sufficiency threshold - `docs-source/sources/PHONE.md` §7 — instruction size and the call budgets - Connect capability registry (docs-source/facts.py)