Prompt injection
Prompt injection is text inside content an agent reads that is written to be obeyed as an instruction. A model has no reliable way to tell the operator's instructions from a sentence in the email it is summarising, so the defence cannot be a better prompt: it has to be that the model's output is a proposal and every real action passes a check the model cannot influence.
The attack, in its business clothes#
The demonstration version is a document containing 'ignore your previous instructions'. The version that reaches a business is quieter, because it arrives inside something a company would read anyway.
- An enquiry whose signature block ends with a line addressed to the automated assistant, asking it to confirm an account change.
- A supplier's PDF with white-on-white text instructing the reader to treat the attached bank details as verified.
- A calendar invitation whose description asks any agent processing it to forward the thread onward.
- A web page an agent researches for a prospect, carrying a paragraph aimed squarely at whatever crawler reads it.
Notice what none of those need: a vulnerability, a credential, or any access at all. The attacker only needs your agent to read something, which is the thing you built it to do. Prompt injection arrives as ordinary business mail has more of the shapes it takes.
Why prompt-level defences do not hold#
- 'Ignore instructions in the content'
- The model is being asked to classify text as instruction or data using the same faculty an attacker is targeting. It works until the phrasing is unfamiliar.
- Delimiters around untrusted text
- Helpful, and defeated by content that closes the delimiter or simply reads convincingly as the operator's own voice.
- A filter for known attack strings
- Blocks the demonstrations. The interesting attacks are paraphrases and there are unlimited paraphrases.
- A second model checking the first
- Adds cost and another model with the same weakness. It raises the bar; it does not change what happens when the bar is cleared.
Each of those reduces frequency. None of them makes a successful injection harmless, which is the only property worth designing for. The question is not 'can content persuade the model' — assume yes — but 'what can the model cause if it is persuaded'.
Does Connect use a defence against prompt injection?#
Used, as containment rather than detection. Four properties do the work, and none of them depends on the model resisting persuasion.
| Property | Effect on a successful injection |
|---|---|
| Actions are tools, not text | Persuaded text is a proposal. The action runs only if a tool exists for it and the tool's own checks pass — see tool calling. |
| Workspace resolves from the session | Instructions to read another workspace's data cannot succeed: the kernel and row-level security filter by the session's workspace, never by an argument. |
| Withheld capabilities | Pricing and clearing a do-not-contact entry are outside the Assistant's tool set. There is nothing to persuade it into. |
| Autonomy holds the irreversible | A workspace on ask-before-send has every outbound reply reviewed by a person before it leaves, whatever produced it. |
Screen context supplied to the Assistant is checked against the database before it is trusted, which closes the neighbouring trick of getting a value in front of the model and hoping it becomes an argument to a write. And because the audit trail records the decision and the rule behind it, an attempt that was refused leaves a record instead of a silence.
What a workspace can do#
Keep the expensive channels on ask-before-send while you learn what arrives.
Result Anything an injection produces is a draft a person reads, not a message a customer receives.
Treat a summary of untrusted content as untrusted.
Result The injection survives summarisation, and a person reading a tidy paragraph is less suspicious than one reading the original.
Keep sensitive changes off the agent's path entirely.
Result Bank details, payment terms and access changes belong with a human and a second channel, regardless of how convincing the request looked.
Questions#
Can an injected instruction reach another customer's data?
No. The workspace is resolved from the session before any query runs, and the filtering happens three times — the tenant allowlist, the ORM kernel, and row-level security in PostgreSQL. An instruction to read across workspaces has nothing to act on, because there is no argument that selects a workspace.
Is this different from jailbreaking?
Yes. Jailbreaking is a user trying to make a model exceed its own rules. Injection is a third party placing instructions in content the agent processes on someone else's behalf — the victim never sees the payload, which is why user-side safeguards do not address it.
Should we stop the agent reading attachments?
That trades most of the value for part of the risk. The better position is that reading is safe and acting is gated: files go through one service with formats checked and dangerous declarations refused, and anything irreversible waits for a person.