Choosing a model
Choose for the shape of the work, not for a benchmark: the latency budget, whether the input is audio or text, how much instruction the model has to hold, and what a wrong answer costs. Connect sets a default per job — one path for drafting and reasoning, a separate speech-to-speech model on the realtime voice path — and a workspace changes behaviour by changing the instructions and budgets around the model, not by picking one from a list.
Four axes, in the order they bind#
Most model comparisons rank quality first because quality is the easy thing to score. In production the binding constraint is usually one of the other three, and a model that wins on quality while losing on the binding axis is not a candidate at all.
| Axis | The question | What it rules out |
|---|---|---|
| Modality | Is the input audio or text? | A text model cannot sit on a live call without a recogniser and a synthesiser in front of it, and each one adds a turn of delay. |
| Latency budget | How long may the first token take? | On a phone call the budget is roughly two and a half seconds to a spoken reply; anything whose first token lands after that is out, whatever it writes. |
| Instruction capacity | How much must it hold and honour at once? | A model that follows the first half of a long instruction block is worse than a smaller one that follows all of a short block. |
| Cost of being wrong | Who sees the output before it leaves? | Where autonomy is autonomous, an unreviewed answer reaches a customer, so the tolerance is narrower than for a draft a person will read. |
Measured, not assumed#
The realtime voice work produced numbers that decide these questions outright. First token on the Flash Live model measured 0.6 s; on the native-audio model of the previous generation it measured 1.2–2.2 s, which is most of a conversational turn spent before a word is spoken. Median reply on the best recorded call was 3.3 s. About 60% of the remaining latency was traced to the model rather than to anything a setting could change.
Prompt size moves the same needle. An instruction block of about 7,500 characters produced a first token in 1.2–1.8 s; at 9,600 characters the same work took 2.3–3.4 s. That is why the budgets are written down as character counts — 2,000 characters of Knowledge and four facts, 700 of memory, 600 for the contact block — rather than left to grow.
Does Connect use a model picker?#
No, and this is deliberate. There is no screen where a workspace selects a model. Connect chooses the provider and model per job in one place — llm.py owns model calls and provider choice — and the levers a workspace is given are the ones that change outcomes without changing the contract: what Knowledge is supplied, what memory is set, what autonomy mode the channel runs in, and what the voice profile says.
The reason is testability. A workspace-selectable model turns every support question into a matrix, and turns the regression suites into a sampling exercise. A single default per job means a behaviour report reproduces, and a fix ships to everybody at once.
Two constraints ride along with the choice and are worth knowing. On the realtime voice path the model is fixed for the life of a session — a live call cannot be moved to a different model mid-conversation. And some capabilities are properties of the model, not of the configuration: where a finding is model-limited, no setting in Connect fixes it.
What moving a model actually costs#
- The instructions do not port. Wording that a model honoured is a fact about that model. A move re-opens every refusal rule, every length instruction and every style block.
- The measurements reset. First token, median reply and the share of latency the model owns are per-model figures. Quoting the old ones after a move is the most common way a docs page becomes false.
- Cost arithmetic changes shape. Audio tokens are priced separately from text — on the voice path, $3.00 per million in and $12.00 per million out against $0.75 and $4.50 for text — so a change of modality is a change of unit, not a discount.
- Failure modes are not shared. A model that refuses in an unexpected place, or that ends a session on a parameter the previous one accepted, produces incidents that look like network faults until somebody reads the close code.
Questions#
Can a workspace ask for a specific model?
There is no setting for it. The behaviour a workspace can change is the material the model is given and the permission it works under — Knowledge, memory, standing instructions, autonomy mode and, on voice, the profile and regional style. Those move outcomes further than a model swap usually does.
Is a bigger context window the same as a better model for this?
No. Capacity to accept a long input is not the same as fidelity across it, and on a live call a longer prompt is directly a slower first token — the measured jump from 7,500 to 9,600 characters is over a second. Connect spends its budget on selection rather than on volume.
Does Connect train a model on workspace data?
No. Nothing a workspace stores is used to train a model. Knowledge and memory are retrieved and supplied at call time, and what is supplied is bounded by the character budgets above.