Rate limiting
A rate limit caps how much work a caller may cause in a period. You meet them in two directions — providers limit what Connect may send them, and Connect limits what a workspace may consume. A 429 is not a request-level problem to sleep off: it says the rate is wrong, so the fix is fewer requests in flight, not the same requests slightly later.
Four shapes, and what each one feels like#
| Shape | How it behaves | The surprise |
|---|---|---|
| Fixed window | N per clock minute, reset on the boundary | Two full allowances land back to back across a boundary — briefly double the intended rate |
| Sliding window | N across the trailing period | Smoother, and harder to predict when you will be allowed through again |
| Token bucket | Tokens refill at a steady rate; a request spends one | Permits a genuine burst after a quiet spell, then throttles hard when the bucket empties |
| Concurrency cap | At most N in flight, whatever the rate | The one that actually protects a downstream service, because it bounds work rather than arrivals |
Most providers combine a rate cap with a concurrency cap, and the second is usually the one you are hitting. That matters for the fix: a client that slows each request but keeps opening new ones has not reduced concurrency at all.
What a 429 should do to a queue#
The instinctive response — catch the 429, wait a moment, send it again — is close to useless and often harmful. While that one request sleeps, the rest of the queue is still being submitted at the rate that caused the problem, so the limiter keeps refusing and the sleeping retries pile up behind work that has not slowed down.
Reduce the number of requests in flight, immediately and sharply.
Result The pressure actually drops. Halving concurrency does more in one step than any per-request delay.
Honour
Retry-Afterwhere the provider sends one; otherwise back off with randomised, growing delays.Result You return when the far end is ready rather than when your own timer says so — and not at the same instant as every other client.
Shed or defer what is not urgent, and say so in the record.
Result The queue drains the work that matters. Silently dropping the rest is the failure this step exists to prevent.
Recover concurrency slowly, not in one jump.
Result You find the ceiling instead of hitting it again, which is the difference between one incident and a repeating one.
Does Connect use rate limiting?#
Used — and it is worth separating two different mechanisms that both produce a refusal. A *rate limit* is technical: requests per period, protecting a service. An *allowance* is commercial: what a plan includes over a day, enforced by metering and the daily ledger. They fail differently and are fixed in different places.
| Rate limit | Plan allowance | |
|---|---|---|
| Period | Seconds | A day |
| Set by | The service being protected | The workspace's plan |
| When it binds | Bursts and loops | Sustained volume |
| What you see | A refusal to accept the request now | A held action that goes when the allowance resets |
| Who has none | Nobody | The Owner — the operator's workspace has no plan and no gates |
The allowance is why a held email draft can be approved and still not go: the approval is recorded, the send waits, and the item stays in Needs You until it leaves. Voice has its own ceiling of the same family — every call is priced on audio tokens, and a call the budget cannot cover is stopped rather than allowed to run past it.
Limiting your own callers#
- Limit by identity, not by address. Many callers share an address behind a proxy, and one caller can present many. Key on the authenticated account.
- Say what happened. A refusal that does not distinguish 'too fast' from 'not permitted' from 'plan exhausted' sends an integrator to the wrong fix. Connect's answer to that is a documented error shape rather than a bare status.
- Protect the expensive path first. Not all requests cost the same. A cheap read and a model call do not belong under one number.
- Measure the cost you are actually limiting. Query cost here is measured in statements rather than wall-clock for exactly that reason: composing on the server took Home from 133 statements to 15, flat with page size, and a limit written against the old shape would have been protecting the wrong thing.
Questions#
Is a 429 a fault or a working system?
A working one, doing its job. The fault would be accepting the work and collapsing under it — which fails everybody rather than slowing one caller. Treat it as feedback about your rate, not as an error to route to a support queue.
Does an allowance running out lose the work?
No. A held action waits — an approval is recorded and the send goes when the allowance permits it. It stays visible in Needs You meanwhile, so a person can see that something is waiting on a limit rather than on them.
Do the Owner and a customer hit the same ceilings?
The same technical ones; not the same commercial ones. Every capability is available to both audiences over one implementation, and the only difference is that a customer's use is bounded by their plan's allowances while the operator's workspace has no plan.