# Testing retrieval

Search runs the same selection a reply runs, for a question you type, and returns the passages with their score, source, authority and section. It is the fastest way to find out whether a document is reachable before a customer finds out for you. It does not write an answer and it does not prove the model will use what it returns.

- **Status:** Available
- **Audience:** both
- **In the app:** #/knowledge-sources-advanced
- **Last verified:** 2026-09-10
- **Canonical:** https://connectbyjbrh.com/docs/knowledge/retrieval-test/

## Running one

1. Type the question the way a customer would type it, not the way the document phrases it.
   - Result: The whole point is to test the words that will actually arrive. Searching for the document's own heading proves nothing.
2. Pick the scope that matches the work — a customer reply, support, prospecting, qualification or internal.
   - Result: The eligible set narrows exactly as it would in the real path. A source scoped `internal` will not appear under `customer_reply`, which is usually the discovery people need.
3. Read the rows.
   - Result: Each carries a score, the source title, its authority, the section heading, the passage index and the passage text. Nothing is summarised — you are looking at what would be handed to the model.

> **Note** The query must be at least two characters, and the number of rows is capped. Internal scope is always included alongside whatever you pick, which mirrors the real path rather than flattering it.

## Reading the result honestly

| Column | What to check |
|---|---|
| Score | Relative, not absolute. The best match for that question is anchored near 1; what matters is the gap between the first row and the fifth. |
| Authority | Whether the top rows can prove anything. Five `reference` rows will not answer a warranty question however well they match. |
| Source title | Whether one document dominates, or the same statement arrives from three. |
| Section | Whether the passage came from the heading you expected — a wrong section usually means the document lost its structure in extraction. |
| Passage index | How deep into the document the match sits. Everything matching at index 30 and beyond suggests the front of the document says nothing useful. |

A healthy result for an important question looks like this: two or three rows from different sources, the top one carrying fact authority, a visible drop in score after the third, and section names you recognise. A flat list of eight near-equal `reference` rows means the bank has breadth and no authority, which is the state that produces confident, unsupported prose.

## What it cannot tell you

- It does not run the sufficiency check. A row can rank first and still be below the threshold that lets a commercial fact be stated.
- It does not apply the live-call budget. Eight rows here may become four entries and 2,000 characters on the phone.
- It does not include memory, the contact block or standing instructions, all of which sit beside Knowledge in a real brief.
- It does not prove the model will use the passage well. Retrieval is the half of grounding that can be tested mechanically; the wording of the answer is not.

For the parts it cannot cover, the honest check is the other direction: make the workspace do the work and then read what was recorded. An answer's citations are kept with the reply, so [tracing one sentence back](/docs/workflows/knowledge-audit/) is a question about the record rather than about retrieval.

## The customer-side preview is a simpler thing

In a customer workspace the equivalent lookup lists sources whose title or text contains what you typed. It answers *is this anywhere in my Knowledge* rather than *what would rank*, and every row comes back with the same nominal score. That is a genuine difference between the two audiences and it is worth knowing before reading too much into a flat result.

The reply path in a customer workspace still ranks properly. It is only this preview that is simpler, and the practical consequence is that a customer workspace tests coverage here and tests ranking by looking at what real replies cited.

## Questions

### How often is this worth doing?

After adding or re-indexing anything, and before any period when volume matters — a campaign going out, a launch, a busy Monday. It takes seconds and it catches the two failures that are otherwise invisible until a customer hits them: a source stuck in review, and a scope that excludes the work you care about.

### The right document is there but ranks fourth. Is that a problem?

On email, rarely — eight passages are taken. On a call it matters, because only four entries travel. Either tighten the source so the answer's words appear in it, or promote it from `reference` to the authority it actually deserves, which also earns it a small ranking adjustment.

### Can I test what a specific customer would get?

Not directly — search takes a question and a scope, not a person. The contact-specific parts of a brief come from memory and the relationship record rather than from Knowledge, so they are checked in the memory viewer instead. See [seeing what Connect knows](/docs/memory/viewing/).

## Related

- [Retrieving Knowledge for an answer](https://connectbyjbrh.com/docs/knowledge/retrieval/)
- [Knowledge sources](https://connectbyjbrh.com/docs/knowledge/sources/)
- [Knowledge for the Owner and for a customer](https://connectbyjbrh.com/docs/knowledge/knowledge-owner-tenant/)
- [Seeing what Connect knows](https://connectbyjbrh.com/docs/memory/viewing/)
- [Connect said it does not know](https://connectbyjbrh.com/docs/troubleshooting/knowledge-insufficient/)

## What this page is based on

- `backend/app/api.py` — the Knowledge search endpoint and its parameters
- `backend/app/tenant_unified_ui.py` — the customer-side Knowledge lookup
- `backend/app/knowledge_bank.py` — the retrieval it calls
- Connect capability registry (docs-source/facts.py)