# Knowledge sources

A Knowledge source is one uploaded file or one fetched web page, stored with a SHA-256 hash, an authority level, a set of scopes and a status. SKILL.md, Markdown, plain text, PDF, PNG/JPEG/WebP images and MP4/MOV/WebM video are accepted; anything else is refused at upload. Only a source whose status is `active` is ever read into an answer.

- **Status:** Available
- **Audience:** both
- **In the app:** #/knowledge-sources-advanced
- **Last verified:** 2026-09-10
- **Canonical:** https://connectbyjbrh.com/docs/knowledge/sources/

## What can be added

| Kind | Files | Size ceiling |
|---|---|---|
| `skill` | A file named `SKILL.md` | 5 MB |
| `markdown` | `.md`, `.markdown` | 5 MB |
| `text` | `.txt` | 5 MB |
| `pdf` | `.pdf` | 50 MB |
| `image` | `.png`, `.jpg`, `.jpeg`, `.webp` | 15 MB |
| `video` | `.mp4`, `.mov`, `.webm` | 250 MB |
| `website` | A public URL, fetched on request | n/a — the fetch has its own limits |

The ceilings are configuration rather than physics, and an operator can change them. The extension is not trusted on its own: the first bytes of the file are checked against the format they claim to be, so a spreadsheet renamed `.pdf` is refused rather than half-parsed. Office documents and spreadsheets are not Knowledge source types — those go through [Files and data](/docs/files-data/), which reads a wider set of formats for a different purpose.

## Authority, and why it is asked for at upload

Authority is not metadata that can be corrected later without consequence. Raising a source from `reference` to `verified_fact` re-runs the trust checks — the injection scan and, for fact and instruction levels, the contradiction check against everything already approved. A source cannot become factual authority through a quiet edit to a dropdown.

- `verified_fact` — a statement of what is true about the business now.
- `owner_instruction` — a direction from the person accountable; carries both fact authority and behaviour authority, and ranks highest.
- `playbook` — how to handle a kind of situation. Behaviour only.
- `reference` — background. Useful, never proof.
- `unverified` — kept for context, deliberately not trusted.
- `roadmap` — what is intended rather than what exists, and scored down so it does not surface as though it were current.

> **Careful** A `SKILL.md` file is data like anything else. It becomes instruction authority only when a person deliberately marks it `owner_instruction` or `playbook`. Uploading one does not hand it the controls.

## Scope: which work a source may reach

Scopes keep an internal engineering note out of a customer reply without anybody having to remember to delete it. A source is tagged with one or more of `customer_reply`, `support`, `prospect`, `qualification` and `internal`, or with `all`. Retrieval asks for the scopes that suit the work in hand, and a source that overlaps none of them is not considered at all.

The scopes a piece of work asks for follow from what it is. A complaint or a legal question asks for support material; a new enquiry, a pricing question or a demo request asks for prospect material; internal material is always available to the reasoning and never on its own sufficient to make a customer-facing claim.

## Status, and the four ways a source goes quiet

| Status | Meaning | Read into answers |
|---|---|---|
| `active` | Indexed, scanned, nothing outstanding | Yes |
| `needs_review` | Warnings or conflicts a person has not resolved | No |
| `disabled` | Switched off by a person | No |
| `error` | Extraction or indexing failed; the reason is on the record | No |

There is a fifth, quieter case. Every non-web source is hashed when it is stored, and the hash is re-checked at retrieval time. If the file on disk no longer matches, the source is not read, its status is moved to `needs_review` and the integrity failure is logged. This is the check that catches a file changed underneath the record rather than through it.

## Adding a page from the web

1. Give a public URL rather than a file.
   - Result: The address is resolved and checked before anything is fetched.
2. Wait for the fetch.
   - Result: Private, loopback, link-local, multicast and other non-global addresses are refused — on the original request and on every redirect, because a redirect is where that check is usually evaded. The site's `robots.txt` is honoured.
3. Set authority and scope, as for any other source.
   - Result: The page is stored, cleaned, chunked and scanned exactly like an uploaded file. A web source has no file on disk, so the integrity hash check does not apply to it; re-fetch it when the page changes.

## Questions

### Why was my spreadsheet refused?

Spreadsheets are not one of the seven Knowledge source types. Export the part that is actually reference material as Markdown or text and upload that, or put the workbook in Files where a wider set of formats is read. A 40-column export is rarely good Knowledge in any case — retrieval works on passages of prose, not on rows.

### Can I change a source's authority after uploading it?

Yes, and doing so re-runs the trust checks rather than simply saving the new value. If the contradiction check then finds a conflict with existing approved material, the source moves to `needs_review` and stops answering until somebody resolves it.

### What happens to a source I delete?

The record, its chunks and the stored file are removed, and the deletion is logged. Answers already sent are unaffected — they were sent — but the citation on an old audit record will no longer resolve to a live source, which is itself useful information when tracing an answer.

## Related

- [Knowledge in Connect](https://connectbyjbrh.com/docs/knowledge/)
- [Extracting text from a source](https://connectbyjbrh.com/docs/knowledge/extraction/)
- [How Knowledge is prepared for retrieval](https://connectbyjbrh.com/docs/knowledge/indexing/)
- [Instructions hidden in content](https://connectbyjbrh.com/docs/knowledge/prompt-injection/)
- [Files and data in Connect](https://connectbyjbrh.com/docs/files-data/)
- [Adding a Knowledge source, end to end](https://connectbyjbrh.com/docs/workflows/add-knowledge/)

## What this page is based on

- `backend/app/knowledge_bank.py` — source types, limits, authority, scope and status
- `docs-source/sources/GENERAL.md` §9 — file formats and the file service
- Connect capability registry (docs-source/facts.py)