Knowledge sources
A Knowledge source is one uploaded file or one fetched web page, stored with a SHA-256 hash, an authority level, a set of scopes and a status. SKILL.md, Markdown, plain text, PDF, PNG/JPEG/WebP images and MP4/MOV/WebM video are accepted; anything else is refused at upload. Only a source whose status is active is ever read into an answer.
What can be added#
| Kind | Files | Size ceiling |
|---|---|---|
skill | A file named SKILL.md | 5 MB |
markdown | .md, .markdown | 5 MB |
text | .txt | 5 MB |
pdf | .pdf | 50 MB |
image | .png, .jpg, .jpeg, .webp | 15 MB |
video | .mp4, .mov, .webm | 250 MB |
website | A public URL, fetched on request | n/a — the fetch has its own limits |
The ceilings are configuration rather than physics, and an operator can change them. The extension is not trusted on its own: the first bytes of the file are checked against the format they claim to be, so a spreadsheet renamed .pdf is refused rather than half-parsed. Office documents and spreadsheets are not Knowledge source types — those go through Files and data, which reads a wider set of formats for a different purpose.
Scope: which work a source may reach#
Scopes keep an internal engineering note out of a customer reply without anybody having to remember to delete it. A source is tagged with one or more of customer_reply, support, prospect, qualification and internal, or with all. Retrieval asks for the scopes that suit the work in hand, and a source that overlaps none of them is not considered at all.
The scopes a piece of work asks for follow from what it is. A complaint or a legal question asks for support material; a new enquiry, a pricing question or a demo request asks for prospect material; internal material is always available to the reasoning and never on its own sufficient to make a customer-facing claim.
Status, and the four ways a source goes quiet#
| Status | Meaning | Read into answers |
|---|---|---|
active | Indexed, scanned, nothing outstanding | Yes |
needs_review | Warnings or conflicts a person has not resolved | No |
disabled | Switched off by a person | No |
error | Extraction or indexing failed; the reason is on the record | No |
There is a fifth, quieter case. Every non-web source is hashed when it is stored, and the hash is re-checked at retrieval time. If the file on disk no longer matches, the source is not read, its status is moved to needs_review and the integrity failure is logged. This is the check that catches a file changed underneath the record rather than through it.
Adding a page from the web#
Give a public URL rather than a file.
Result The address is resolved and checked before anything is fetched.
Wait for the fetch.
Result Private, loopback, link-local, multicast and other non-global addresses are refused — on the original request and on every redirect, because a redirect is where that check is usually evaded. The site's
robots.txtis honoured.Set authority and scope, as for any other source.
Result The page is stored, cleaned, chunked and scanned exactly like an uploaded file. A web source has no file on disk, so the integrity hash check does not apply to it; re-fetch it when the page changes.
Questions#
Why was my spreadsheet refused?
Spreadsheets are not one of the seven Knowledge source types. Export the part that is actually reference material as Markdown or text and upload that, or put the workbook in Files where a wider set of formats is read. A 40-column export is rarely good Knowledge in any case — retrieval works on passages of prose, not on rows.
Can I change a source's authority after uploading it?
Yes, and doing so re-runs the trust checks rather than simply saving the new value. If the contradiction check then finds a conflict with existing approved material, the source moves to needs_review and stops answering until somebody resolves it.
What happens to a source I delete?
The record, its chunks and the stored file are removed, and the deletion is logged. Answers already sent are unaffected — they were sent — but the citation on an old audit record will no longer resolve to a live source, which is itself useful information when tracing an answer.