Connect by JBRH Open Connect

Knowledge sources

A Knowledge source is one uploaded file or one fetched web page, stored with a SHA-256 hash, an authority level, a set of scopes and a status. SKILL.md, Markdown, plain text, PDF, PNG/JPEG/WebP images and MP4/MOV/WebM video are accepted; anything else is refused at upload. Only a source whose status is active is ever read into an answer.

Status
Available What this means
Audience
both
In the app
#/knowledge-sources-advanced
Last verified
Product version
6.3.2

What can be added#

KindFilesSize ceiling
skillA file named SKILL.md5 MB
markdown.md, .markdown5 MB
text.txt5 MB
pdf.pdf50 MB
image.png, .jpg, .jpeg, .webp15 MB
video.mp4, .mov, .webm250 MB
websiteA public URL, fetched on requestn/a — the fetch has its own limits

The ceilings are configuration rather than physics, and an operator can change them. The extension is not trusted on its own: the first bytes of the file are checked against the format they claim to be, so a spreadsheet renamed .pdf is refused rather than half-parsed. Office documents and spreadsheets are not Knowledge source types — those go through Files and data, which reads a wider set of formats for a different purpose.

Authority, and why it is asked for at upload#

Authority is not metadata that can be corrected later without consequence. Raising a source from reference to verified_fact re-runs the trust checks — the injection scan and, for fact and instruction levels, the contradiction check against everything already approved. A source cannot become factual authority through a quiet edit to a dropdown.

  • verified_fact — a statement of what is true about the business now.
  • owner_instruction — a direction from the person accountable; carries both fact authority and behaviour authority, and ranks highest.
  • playbook — how to handle a kind of situation. Behaviour only.
  • reference — background. Useful, never proof.
  • unverified — kept for context, deliberately not trusted.
  • roadmap — what is intended rather than what exists, and scored down so it does not surface as though it were current.

Scope: which work a source may reach#

Scopes keep an internal engineering note out of a customer reply without anybody having to remember to delete it. A source is tagged with one or more of customer_reply, support, prospect, qualification and internal, or with all. Retrieval asks for the scopes that suit the work in hand, and a source that overlaps none of them is not considered at all.

The scopes a piece of work asks for follow from what it is. A complaint or a legal question asks for support material; a new enquiry, a pricing question or a demo request asks for prospect material; internal material is always available to the reasoning and never on its own sufficient to make a customer-facing claim.

Status, and the four ways a source goes quiet#

StatusMeaningRead into answers
activeIndexed, scanned, nothing outstandingYes
needs_reviewWarnings or conflicts a person has not resolvedNo
disabledSwitched off by a personNo
errorExtraction or indexing failed; the reason is on the recordNo

There is a fifth, quieter case. Every non-web source is hashed when it is stored, and the hash is re-checked at retrieval time. If the file on disk no longer matches, the source is not read, its status is moved to needs_review and the integrity failure is logged. This is the check that catches a file changed underneath the record rather than through it.

Adding a page from the web#

  1. Give a public URL rather than a file.

    Result The address is resolved and checked before anything is fetched.

  2. Wait for the fetch.

    Result Private, loopback, link-local, multicast and other non-global addresses are refused — on the original request and on every redirect, because a redirect is where that check is usually evaded. The site's robots.txt is honoured.

  3. Set authority and scope, as for any other source.

    Result The page is stored, cleaned, chunked and scanned exactly like an uploaded file. A web source has no file on disk, so the integrity hash check does not apply to it; re-fetch it when the page changes.

Questions#

Why was my spreadsheet refused?

Spreadsheets are not one of the seven Knowledge source types. Export the part that is actually reference material as Markdown or text and upload that, or put the workbook in Files where a wider set of formats is read. A 40-column export is rarely good Knowledge in any case — retrieval works on passages of prose, not on rows.

Can I change a source's authority after uploading it?

Yes, and doing so re-runs the trust checks rather than simply saving the new value. If the contradiction check then finds a conflict with existing approved material, the source moves to needs_review and stops answering until somebody resolves it.

What happens to a source I delete?

The record, its chunks and the stored file are removed, and the deletion is logged. Answers already sent are unaffected — they were sent — but the citation on an old audit record will no longer resolve to a live source, which is itself useful information when tracing an answer.