Agent orchestration
Orchestration is the part of an agent system that decides what runs, in what order, how often, and what happens when a step fails. It is ordinary engineering rather than model work, and it is where most agent systems that misbehave in production are actually broken — usually because two schedulers were allowed to reason about the same record at the same time.
Three shapes, and when each fits#
| Shape | Who decides the next step | Fits |
|---|---|---|
| Fixed sequence | The code | Work whose steps are known: fetch, normalise, classify, draft, decide |
| Model-directed | The model, one tool call at a time | Open-ended requests where the next step depends on what the last one returned |
| Queue drain | A scheduler, on a clock | Dated commitments and retries that must survive a restart |
Connect runs all three, in different places, and the boundaries between them are deliberate. The mail engine is a fixed sequence because its steps do not vary. The Assistant is model-directed because a person's request genuinely might need any of sixty-six tools. Follow-ups drain on a tick because a commitment made on Tuesday has to be kept on Friday whether or not anyone opened the application in between.
Does Connect use orchestration?#
Used, throughout. runner.py holds the periodic work and agent.py holds the loop that thinks about one message; the follow-up queue in followups.py spans email, WhatsApp, SMS, phone, any and task. Two behaviours are worth quoting because they show the shape of the rules rather than just their existence.
runner.chase_phone drains phone follow-ups two per tick, each through the same gates as any other outbound call. An outcome is recorded as call placed <id>, awaiting approval <id> or refused: <reason>. A line that is not ready is retried in an hour. A row more than 24 hours late is closed as missed: and never rung — a call-back a day late is worse than none, so the queue is allowed to give up rather than pretend.
The task channel is the interesting exception: no drain ever sends it, because it is work for a person. It appears under 'For a person to do' on the follow-ups screen. An orchestrator that can only produce machine actions ends up inventing a machine action for every commitment.
The single-thinker rule#
Only one loop may think. Fetching, normalising and bridging a provider's messages into the canonical records can happen in several places; deciding what to *do* about a message happens in exactly one. Connect's periodic tenant tick fetches and bridges and stops there — it does not process a message — and the runner's own pass is what reasons about pending work.
The reason is not tidiness. Two loops reading the same pending thread produce two drafts, two approval items, and eventually two replies to one customer, and the second is not a duplicate anyone notices until it has already gone. Every mitigation short of a single owner — locks, timestamps, deduplication on send — is a race with a smaller window rather than no race. Why only one loop may think has the longer version.
Failure, retry and the cost of getting it wrong#
- Retry without idempotency doubles the side effect. A send that timed out may have succeeded; the retry needs a key the provider will recognise. See idempotency and retries and backoff.
- Unbounded retry is an outage amplifier. A provider returning errors under load gets more traffic from a naive scheduler, not less.
- A queue with no give-up rule grows silently. The 24-hour rule exists so that a stalled line produces a closed row with a reason instead of a call at the wrong moment.
- A step that partially succeeded is the hard case. Recording what completed and what did not — rather than a single success flag — is what makes the next run able to resume instead of repeat.
None of this is model behaviour, which is why an agent system's reliability usually improves more from scheduler work than from prompt work. The model decides what a good reply says. The orchestrator decides whether it is sent once.
Questions#
Is a multi-agent system better than one orchestrator?
Sometimes, for genuinely independent work. It is worse whenever the agents share state, because coordination cost and duplicate-action risk both rise faster than the parallelism helps. Connect keeps one thinker over the shared records and uses separate workers only where the work does not overlap, such as a live call.
What happens to queued work while the runtime is switched off?
It stays queued. Dated commitments do not expire because nothing was running; they are drained when the runtime resumes, subject to the same lateness rule, so a phone follow-up more than a day late is closed rather than rung.
How is a retried step told apart from a new one?
By a stable key carried from the first attempt, not by comparing content. Content comparison fails exactly when it matters — two genuine messages can be identical, and one message can be retried with a different formatting.