# Retrieval-augmented generation

Retrieval-augmented generation means finding relevant material first and generating an answer from it, instead of asking a model to answer from what it learned in training. The pattern is a retrieval problem wearing a generation costume: almost every RAG system that disappoints does so because the retrieval step returned the wrong passage, not because the model wrote it badly.

- **Status:** Reference
- **Audience:** both, developer
- **In the app:** #/knowledge
- **Last verified:** 2026-09-10
- **Canonical:** https://connectbyjbrh.com/docs/technology/rag/

## The pattern, stripped to its parts

1. INGEST — material is taken in and normalised to text
2. SEGMENT — it is broken into units small enough to be selected individually
3. INDEX — each unit is made findable, by whatever mechanism
4. RETRIEVE — the units relevant to this question are selected
5. ASSEMBLE — the selected units are placed in the prompt with an instruction binding the answer to them
6. GENERATE — the model answers from what it was given
7. ATTRIBUTE — the answer carries what it was based on

Only the third and fourth steps involve any technology choice. The popular one is an embedding model plus a vector store, where similarity in a numeric space stands in for relevance. It is not the only one, and for a corpus that is small, structured and owned by the workspace it is often not the best one.

## Does Connect use retrieval-augmented generation?

**Used** — and the honest form of that answer needs one sentence more. **Connect implements retrieval and grounding over the Knowledge a workspace supplies without a vector database and without embedding search.** There is no embedding index, no similarity threshold to tune, and no separate vector service in the deployment.

What runs instead is selection over records the workspace already owns. Knowledge sources and the Facts extracted from them are ordinary rows, scoped to a workspace and filtered by the same isolation that protects every other record. Retrieval picks from those by structure and relevance to the message at hand, and the result is capped hard: on the realtime voice path the Knowledge block is **2,000 characters and at most 4 facts**, alongside **700 characters** of memory and a **600-character** contact block.

The trade is deliberate and it is stated rather than hidden. A vector index would find a paraphrase that shares no words with the question; structural selection will sometimes miss one. In exchange the retrieval is explainable — you can see exactly which fact was used and why — the deployment carries no extra stateful service, there is no re-embedding cost when a document is edited, and there is no second copy of a customer's material living outside the isolation model. [Grounding an answer without pretending to have a vector database](/research/grounding-without-a-vector-database/) works through the decision.

> **Careful** If you are comparing products, ask what happens when retrieval finds nothing rather than which index is used. A system that answers anyway has the expensive failure regardless of how sophisticated its search is.

## Where RAG systems actually fail

| Failure | What it looks like | Where the fix belongs |
|---|---|---|
| Wrong passage retrieved | A confident, sourced, irrelevant answer | Retrieval and segmentation, never the prompt |
| Segments cut badly | An answer missing the condition that followed it in the document | Segment on meaning — a clause, a policy — not on a character count |
| Nothing retrieved, answer produced anyway | Fluent invention with no source | The instruction and the permitted 'not covered' outcome |
| Stale material | Correctly grounded on last year's terms | Review dates and ownership of sources |
| Conflicting sources | One of two contradictory answers, silently chosen | Surfacing the conflict to a person |
| Too much retrieved | Slower, dearer, and less precise than retrieving less | A budget, enforced |

Note that five of those six live outside the model. Swapping the model rarely moves a RAG system's accuracy much; fixing segmentation usually does.

## How it differs from fine-tuning and from memory

**Retrieval** — Material stays outside the model, is selected per question, and can be edited or deleted with immediate effect on the next answer.
**Fine-tuning** — Behaviour is baked into weights. It teaches style and format well and facts badly, and a correction means retraining. Connect does not train a model on customer data.
**Memory** — Durable knowledge about a business and its people, held in [structured records](/docs/technology/structured-memory/) at four tiers, not in a document store. Retrieval answers 'what does the policy say'; memory answers 'what do we know about this customer'.

## Questions

### Is a vector database required for RAG?

No. Vector search is one way to implement the retrieve step, well suited to a large corpus of unstructured prose where a question may share no vocabulary with the answer. Connect's Knowledge is workspace-owned, comparatively small and already structured, so it selects over records directly and says so rather than implying an index it does not run.

### What stops one workspace's Knowledge reaching another's answers?

The same three layers that protect every other record — the tenant allowlist, the workspace kernel in the ORM, and row-level security in PostgreSQL. Keeping Knowledge as ordinary scoped rows rather than in a separate index is part of why there is no fourth place for that to go wrong.

### How do I make retrieval find more of what I mean?

Write the source the way the question will be asked: one topic per section, the conditions next to the claim they qualify, and the words a customer would use rather than internal shorthand. That improves any retrieval mechanism, and it is the only lever that improves all of them at once.

## Related

- [Grounding](https://connectbyjbrh.com/docs/technology/grounding/)
- [Knowledge base](https://connectbyjbrh.com/docs/technology/knowledge-base/)
- [Knowledge in Connect](https://connectbyjbrh.com/docs/knowledge/)
- [Context window](https://connectbyjbrh.com/docs/technology/context-window/)
- [Grounding an answer without pretending to have a vector database](https://connectbyjbrh.com/research/grounding-without-a-vector-database/)

## What this page is based on

- Connect capability registry (docs-source/facts.py) — knowledge_sources, facts_editor, MEASURED grounding budgets
- Connect architecture source pack — docs-source/sources/GENERAL.md §2 and §4, isolation and knowledge modules
- `backend/app/knowledge_bank.py`, `knowledge.py` — the Knowledge and Fact records