# Multilingual voice

Multilingual voice is three decisions rather than one: which language a call opens in, whether the caller has actually changed language, and which script a reply should be written in. Connect decides all three on evidence — the line's language opens the call, a remembered one is spoken from the first reply until a whole sentence overrides it, and a reply is steered only when its script is one the caller has not used.

- **Status:** Available
- **Audience:** both
- **Channels:** phone
- **In the app:** #/calls
- **Last verified:** 2026-09-10
- **Canonical:** https://connectbyjbrh.com/docs/technology/multilingual-voice/

## What is spoken, how it is written, and where

A standard tag written the usual way carries up to three of them, and collapsing any two is where most multilingual bugs start. `hi-Latn-IN` is Hindi, written in Latin letters, as used in India. Drop the middle part and a system that expects Devanagari decides the caller has switched to English.

**Language** — What is being spoken. Independent of how it is written down.
**Script** — The writing system the transcript happens to use. A transcriber may render speech in a script its speakers rarely write it in.
**Locale / region** — Which variety, and often which voice and which number and date conventions. Two locales can need different pronunciations of the same word.

The distinction has teeth on a phone call because the only text anyone sees is a transcript, and a transcript is a rendering choice. A Telugu reply transcribed in Latin letters is a Telugu reply — the tracker marks it `romanised_script` precisely so it is not counted as an English one.

## Detecting a switch is harder than detecting what is spoken

Identifying what a long, clean utterance is spoken in is close to solved. Deciding that a *conversation* has changed is not, because the evidence arrives a few words at a time and half of it is noise: a mis-heard fragment, a borrowed word, a proper noun, a number read out in whichever tongue the speaker counts in.

Systems that switch eagerly produce the worst failure in this area — the voice answering in one the caller does not speak, then apologising in it. The defensive rules that avoid it are all about raising the evidentiary bar rather than improving the classifier.

- **A whole sentence, not a word.** A remembered preference is only adopted on a full sentence from the caller.
- **A mixed line is not a switch.** A sentence that mixes two languages is how a great many people speak, and treating it as a change makes the voice oscillate.
- **One garbled line does not undo what is established.** Noise is more common than a mid-call switch.
- **A pinned line is never steered.** Where one is configured that way, evidence does not override configuration.

## Does Connect use multilingual voice?

**Used, on both engines, and on evidence rather than on guesswork.** The brief opens the call in the line's language. During the call the `LanguageTracker` appends exactly one chat-context note — `language_steer` — and only when a reply came back in a script the caller has not used. The greeting's script follows what the call actually opens in, not whatever the line's setting says in the abstract.

The voice itself became multilingual on 2026-09-04. Before that every word a caller heard was the carrier's own text-to-speech: two voices and sixteen European locales, with no Indian tongue among them. Sentences now go to a text-to-speech API that offers either one voice for every language steered by a natural-language style instruction, or one voice per locale with a speaking rate, and the audio is served to the carrier as a played file rather than as text for it to read.

A regional speaking-style layer sits above that, and the human voice profiles carry politeness markers — *andi*, *ji*, *-nga*, *po* — and syllable-timed rhythm for Indian English. Promise checking runs in English, Telugu and Hindi: a promised time that shares no time word with the caller's own time-bearing line is refused rather than recorded.

> **Careful** One related behaviour is model-limited and was removed rather than left in. A rendered respect-word cadence rule changed six replies in eight and produced a Telugu answer to an English sentence. Do not spend instruction characters on it.

## Where multilingual calls actually break

| What you hear | Usual cause | Where to look |
|---|---|---|
| The voice opens in the wrong one from the first word | The line's own configuration, not a detection failure — the opening is a setting | The line's setting, and its pinned code |
| It switches after one short caller reply | A one-word or fragmentary utterance treated as evidence | Whether a whole sentence was actually spoken |
| The transcript looks English but the caller was not | Romanised transcription of another tongue | The `romanised_script` marking on those rows |
| It flips back and forth | A mixed-language speaker being read as switching every sentence | Whether the line should be pinned instead |

The general operating rule is that configuration is stronger than inference, and inference is stronger than a single line of text. If a business serves one tongue only, pin it: an evidence rule is a way of coping with uncertainty, not a reason to introduce some.

## Questions

### Can one line handle several languages?

Yes. The call opens in the line's language and can move if the caller gives a whole sentence in another. If a line should never move, pinning its language code turns the evidence rules off for that line entirely.

### Does a mixed-language sentence confuse the voice?

It is explicitly not treated as a language change, which is the behaviour that matters — a mixed line is how many people speak, and a system that switches on it will oscillate for the rest of the call.

### Why did the transcript come back in Latin letters for a non-Latin language?

Transcription chooses a script, and romanising is a common choice. The line is still in the language it was spoken in, and it is marked as romanised so that no part of the pipeline mistakes it for English.

## Related

- [Unicode, scripts and transliteration](https://connectbyjbrh.com/docs/technology/unicode-and-scripts/)
- [Multilingual calling](https://connectbyjbrh.com/docs/phone/multilingual-calls/)
- [Detecting the caller's language](https://connectbyjbrh.com/docs/phone/language-detection/)
- [Text to speech](https://connectbyjbrh.com/docs/technology/tts/)
- [Voice profiles](https://connectbyjbrh.com/docs/technology/voice-profile/)
- [Regional speaking style](https://connectbyjbrh.com/docs/phone/regional-speech/)

## What this page is based on

- `docs-source/sources/PHONE.md` §7 — language on evidence
- `docs-source/sources/PHONE.md` §11 — Connect's own voice and profiles
- `docs-source/sources/PHONE.md` §10 — promise checking across languages
- Connect capability registry (docs-source/facts.py)