# Choosing a model

Choose for the shape of the work, not for a benchmark: the latency budget, whether the input is audio or text, how much instruction the model has to hold, and what a wrong answer costs. Connect sets a default per job — one path for drafting and reasoning, a separate speech-to-speech model on the realtime voice path — and a workspace changes behaviour by changing the instructions and budgets around the model, not by picking one from a list.

- **Status:** Reference
- **Audience:** developer, both
- **Last verified:** 2026-09-10
- **Canonical:** https://connectbyjbrh.com/docs/technology/model-selection/

## Four axes, in the order they bind

Most model comparisons rank quality first because quality is the easy thing to score. In production the binding constraint is usually one of the other three, and a model that wins on quality while losing on the binding axis is not a candidate at all.

| Axis | The question | What it rules out |
|---|---|---|
| Modality | Is the input audio or text? | A text model cannot sit on a live call without a recogniser and a synthesiser in front of it, and each one adds a turn of delay. |
| Latency budget | How long may the first token take? | On a phone call the budget is roughly two and a half seconds to a spoken reply; anything whose first token lands after that is out, whatever it writes. |
| Instruction capacity | How much must it hold and honour at once? | A model that follows the first half of a long instruction block is worse than a smaller one that follows all of a short block. |
| Cost of being wrong | Who sees the output before it leaves? | Where autonomy is `autonomous`, an unreviewed answer reaches a customer, so the tolerance is narrower than for a draft a person will read. |

## Measured, not assumed

The realtime voice work produced numbers that decide these questions outright. First token on the Flash Live model measured **0.6 s**; on the native-audio model of the previous generation it measured **1.2–2.2 s**, which is most of a conversational turn spent before a word is spoken. Median reply on the best recorded call was **3.3 s**. About **60%** of the remaining latency was traced to the model rather than to anything a setting could change.

Prompt size moves the same needle. An instruction block of about 7,500 characters produced a first token in 1.2–1.8 s; at 9,600 characters the same work took 2.3–3.4 s. That is why the budgets are written down as character counts — 2,000 characters of Knowledge and four facts, 700 of memory, 600 for the contact block — rather than left to grow.

> **Note** A benchmark score is a statement about a test set. A first-token measurement on your own traffic is a statement about your product. Where the two disagree on this system, the measurement has won every time.

## Does Connect use a model picker?

No, and this is deliberate. There is no screen where a workspace selects a model. Connect chooses the provider and model per job in one place — `llm.py` owns model calls and provider choice — and the levers a workspace is given are the ones that change outcomes without changing the contract: what Knowledge is supplied, what memory is set, what autonomy mode the channel runs in, and what the voice profile says.

The reason is testability. A workspace-selectable model turns every support question into a matrix, and turns the regression suites into a sampling exercise. A single default per job means a behaviour report reproduces, and a fix ships to everybody at once.

Two constraints ride along with the choice and are worth knowing. On the realtime voice path the model is fixed for the life of a session — a live call cannot be moved to a different model mid-conversation. And some capabilities are properties of the model, not of the configuration: where a finding is model-limited, no setting in Connect fixes it.

## What moving a model actually costs

- **The instructions do not port.** Wording that a model honoured is a fact about that model. A move re-opens every refusal rule, every length instruction and every style block.
- **The measurements reset.** First token, median reply and the share of latency the model owns are per-model figures. Quoting the old ones after a move is the most common way a docs page becomes false.
- **Cost arithmetic changes shape.** Audio tokens are priced separately from text — on the voice path, $3.00 per million in and $12.00 per million out against $0.75 and $4.50 for text — so a change of modality is a change of unit, not a discount.
- **Failure modes are not shared.** A model that refuses in an unexpected place, or that ends a session on a parameter the previous one accepted, produces incidents that look like network faults until somebody reads the close code.

## Questions

### Can a workspace ask for a specific model?

There is no setting for it. The behaviour a workspace can change is the material the model is given and the permission it works under — Knowledge, memory, standing instructions, autonomy mode and, on voice, the profile and regional style. Those move outcomes further than a model swap usually does.

### Is a bigger context window the same as a better model for this?

No. Capacity to accept a long input is not the same as fidelity across it, and on a live call a longer prompt is directly a slower first token — the measured jump from 7,500 to 9,600 characters is over a second. Connect spends its budget on selection rather than on volume.

### Does Connect train a model on workspace data?

No. Nothing a workspace stores is used to train a model. Knowledge and memory are retrieved and supplied at call time, and what is supplied is bounded by the character budgets above.

## Related

- [What a model call costs](https://connectbyjbrh.com/docs/technology/llm-cost/)
- [Context window](https://connectbyjbrh.com/docs/technology/context-window/)
- [Speech-to-speech models](https://connectbyjbrh.com/docs/technology/speech-to-speech/)
- [Voice latency](https://connectbyjbrh.com/docs/technology/voice-latency/)
- [What every character of a prompt costs on a live call](https://connectbyjbrh.com/research/prompt-size-and-first-token/)
- [Findings a setting cannot fix](https://connectbyjbrh.com/research/model-limited-findings/)

## What this page is based on

- Connect capability registry (docs-source/facts.py) — MEASURED
- Connect source pack (docs-source/sources/GENERAL.md §4) — `llm.py` owns model and provider choice
- Connect source pack (docs-source/sources/GENERAL.md §11) — measured facts