Detecting the caller's language
Detection here means deciding, on evidence, that the caller is speaking something other than what the call opened in. There is no single bar. A whole sentence is needed to override a remembered language, or to move a call Connect knows nothing about; two or three words are enough where the contact's city or state makes a language likely; a pinned line never moves at all.
The bar moves with what is already known#
Getting this wrong is cheap in code and expensive on a call. A system that flips on weak evidence answers a Hindi speaker's English question in Hindi, then flips back, and the caller spends the conversation wondering who they are talking to. A system that never flips answers everybody in the line's default, which is a smaller failure and a recoverable one — the caller can ask. voice_region.plan resolves that tension by setting a different bar in each of four tiers.
| What Connect knows | What the call opens in | What moves it |
|---|---|---|
| A pinned language, switching not set to follow | The pinned language | Nothing. Any language is understood; the reply stays pinned unless the caller asks for another |
A remembered preferred_language | The greeting, then the remembered language from the first reply | A whole sentence in another language. Never a switch mid-sentence, and never mixed scripts unless the caller mixes them |
No memory, but the contact's city or state maps to a row in REGIONS | The line's language — the regional one is not assumed | Two or three words in the regional language, mixed with English or not. From then on the reply is in it and stays there |
| Nothing on file beyond, at most, a country | The line's language | A whole sentence in another language; then the call stays in it |
The likely tier is the one that catches people out, because its bar is deliberately *lower* than a sentence. A caller in Warangal who drops three Telugu words into an English question has said enough. Nothing is offered in return and no sample phrase appears in the brief: on a production call on 2026-09-10 an earlier brief's line about adding one short polite Telugu phrase was read out to the caller word for word, and because a steer cannot reach a Gemini 3.1 session mid-call, the opening line is the whole lever.
Mid-call, LanguageTracker appends its note only when a reply has come back in a script the caller has not used. Note the direction of that condition: the trigger is a *mismatch* that already happened, not a prediction.
What never counts, and what counts only sometimes#
| What was heard | How it is treated |
|---|---|
| A single word | "Haan", "sari", "okay" — greeting and agreement words travel between languages, so one of them will not override a remembered language or move a call with nothing on file |
| A mixed line | Where the language is remembered or unknown, Hinglish is a register, not a request. Where the contact's city or state makes a language likely, that same mixed line is exactly the evidence the brief asks for |
| A Telugu reply written in Latin letters | romanised_script — the transcriber's alphabet is not the speech. This one produced real false switches before it was handled |
| One garbled line | Noise, a bad handset or a mis-transcription. What is already established survives a single bad line |
The third row is the subtle one and worth dwelling on. Transcription systems routinely write Indian-language speech in Latin script. A detector comparing scripts rather than speech sees Latin letters, concludes English, and steers a Telugu conversation into English on the strength of a formatting decision made by a different system.
What actually happens when the bar is met#
- One chat-context note —
language_steer— is appended. One, not one per turn. - It reaches the model through
agent.update_chat_ctx, which takes a user-role turn without generating a reply from it. - The next reply is produced with that note in context, alongside everything else the conversation carries.
The delivery mechanism matters more than it sounds. The alternative, update_instructions, resends the whole prompt — a real cost when instruction size is measurably a latency control — and it is kept for supervisor guidance only. A steering note is a small thing and travels as one.
Pinning switches detection off#
A pinned code in the Voice Lab is an engine setting, and a pinned call is never steered. That is absolute rather than a strong preference: no amount of contrary evidence produces a note. Use it when a line exists to be answered one way — a regional service number, a script that has been approved once — and leave it unset when the line's job is to meet whoever rings.
The greeting is the other fixed point, because it is pre-synthesised audio decided before the caller has said anything. Multilingual calling covers how those two settings and the line's own interact at the start of a call.
What is verified today#
The rules above were reconciled on 2026-09-10 against backend/app/voice_region.plan at f5ea518, read from the code rather than the module docstring beside it — that docstring still says a likely language is offered once, lightly, and the code stopped offering anything at all. If you are checking this page against the source, read plan() itself. The transcript artefacts — a romanised row, one garbled line — are each the result of a real false switch, and the reasons they were rejected do not stop being true.
Questions#
The caller said one sentence in Hindi and Connect stayed in English. Why?
Check whether the code is pinned on that line — a pinned call is never steered, whatever is heard. If it is not pinned, check whether the line was running on a session that accepts mid-call notes; on the immutable default model a steering note cannot be delivered, so the call runs on whatever the opening brief established.
Can I make detection more sensitive?
There is deliberately no sensitivity dial: the bar in each tier is behaviour, not a setting. The way to make a call more responsive to a regional language is to put the contact's city or state on file, which moves it into the likely tier where two or three words are enough. To make it less responsive, pin the line.
Does a switch mid-call change the transcript?
The transcript records what was said, including a reply in a script the caller had not used — that mismatch is exactly the trigger. A romanised transcript row is not evidence of anything about the speech, which is why romanised_script exists as a distinct case.