How Knowledge is prepared for retrieval
Extracted text is split into heading-aware passages of about 1,800 characters with 180 characters of overlap, up to 240 per source, and each is stored as an ordinary row alongside its section name. Connect has no vector database. Where embeddings are configured, the vector sits in a column on that row and is one of two scores; there is no approximate nearest-neighbour index anywhere in the path.
Say the mechanism plainly#
Products in this category usually describe their grounding as a vector store, and readers now assume embedding search whenever the word *retrieval* appears. That assumption is wrong here, so it is worth stating flatly rather than leaving to inference: a workspace's Knowledge lives in the same PostgreSQL database as everything else, one row per passage, under the same row-level security. Nothing is copied into a separate search service.
The consequence is that every eligible passage is scored on every question — a full pass, not an approximate lookup. At the size a single business's reference material actually reaches, that is both fast enough and considerably easier to reason about, because there is no index to fall out of date and no second copy of the data to isolate. The engineering note grounding without a vector database sets out where that stops being true.
How the text is divided#
The splitter is heading-aware. Markdown headings become section names, which is why extraction is asked to preserve headings and why a well-structured document retrieves better than the same words as one block. Text before any heading, or a document with none, is filed under *Document*.
- Within a section, whole paragraphs are packed together until adding the next one would exceed about 1,800 characters.
- When a passage closes, its last 180 characters are carried into the next one, so a sentence spanning the join is findable from either side.
- A single paragraph longer than the limit — a transcript block, a wide table — is hard-split at the same step, keeping the overlap.
- At most 240 passages are kept per source. A document that needs more is a document that should be several sources.
- The section name is stored with the passage and shown in citations, so a person tracing an answer lands on the heading it came from.
What is stored on each passage#
| Field | What it is for |
|---|---|
| Source | Which source it came from — the link to authority, scope and status |
| Index | Its position in the document; the second half of the citation |
| Section | The heading it sat under, up to 300 characters |
| Content | The passage text itself |
| Token count | Used for scoring; a long passage is not rewarded for length alone |
| Embedding | A vector, or empty. Optional, and empty on the customer path |
The source record itself gains a passage count, a timestamp for when the work finished, a short summary of the opening, the list of scan warnings and the list of detected conflicts. If either list is non-empty the source lands in needs_review rather than active, which is how a problem announces itself instead of quietly changing what Connect says.
Where embeddings do and do not appear#
When semantic embeddings are switched on and a model key is configured, each passage is embedded as it is stored and the vector is kept in the column above. Retrieval then blends cosine similarity with the lexical score. When no passage in the eligible set carries a vector, the question is not embedded at all and ranking is purely lexical.
A customer workspace reports its embedding model as lexical and its embedding dimensions as zero. This is not a degraded mode that needs fixing; it is the deliberate shape of the customer path, and it means no customer text is sent to an embedding service to make search work.
Questions#
Is there any vector database in Connect?
No. There is no vector store, no approximate nearest-neighbour index, and no separate search cluster. Passages are rows in the main database, and where embeddings exist they are a column on those rows used for one of two scoring terms.
Why 1,800 characters?
It is large enough that a passage usually carries a complete thought — a clause, a specification, a policy — and small enough that four of them fit inside the character budget a live call allows for Knowledge. The number is configuration, and the overlap keeps a split sentence findable.
Does re-indexing change my citations?
It can. Re-indexing deletes the existing passages and rebuilds them, so positions can shift if the extracted text changed. The source identifier in a citation is stable; the passage index within it is not, which is worth knowing when auditing an old answer.