TL;DR. ServiceNow AI published on 9 June 2026 a systematic benchmark of frontier ASR models on code-switched speech — conversations where bilingual speakers alternate between two languages mid-sentence. For enterprises deploying voice agents in European multilingual markets, this research formalises a procurement gap that standard vendor datasheets have never tested for.
A recurring failure mode across voice deployments
Voice AI systems are designed, built, and evaluated on clean, monolingual audio. Customers in Brussels, Luxembourg, or Geneva do not speak that way.
The deployment sequence is consistent: a voice agent passes laboratory benchmarks, receives sign-off, goes live in a bilingual market, and encounters code-switching — the natural pattern where a fluent speaker alternates between two languages within a single conversation. Transcription accuracy degrades. The model defaults to the dominant language, mishandles the switch, or returns a low-confidence output at the precise moment the customer provides the most critical information.
ServiceNow AI formalised this gap in research published on Hugging Face on 9 June 2026, under the title Can Voice Agents Handle Bilingual Customers? Benchmarking Frontier ASR on Code-Switched Speech. The research question itself is the signal: the failure mode is systematic, not incidental.
What does code-switching actually cost an enterprise?
When a speaker alternates between two languages mid-sentence, an ASR model benchmarked exclusively on monolingual corpora produces degraded output at exactly that juncture. The published accuracy figures in vendor datasheets do not predict production performance in bilingual markets.
Three deployment scenarios illustrate the exposure.
First: customer service voice agents. A caller opens in Dutch, shifts to French for a legal or technical term, returns to Dutch for the reference number. A model trained only on monolingual Dutch audio has no representation of that switch. The transcription breaks where the interaction matters most.
Second: internal meeting transcription in pan-European organisations. Multilingual teams shift languages for conceptual precision — a term without an equivalent in the current working language triggers a code-switch. Monolingual ASR models classify this signal as noise rather than input.
Third: voice-authenticated workflows. A user enrolled a voice profile in one language. Under cognitive load or in a multilingual environment, they naturally code-switch. An authentication pipeline built on monolingual acoustic models degrades in exactly the scenario where reliability is the stated requirement.
In Belgium, Luxembourg, or Switzerland, these are not edge cases. They describe baseline usage patterns across public services, financial institutions, and pan-European enterprise teams.
What actually drives the pattern?
The root cause is structural. Standard ASR benchmarks — the performance tables vendors publish — use clean, monolingual speech corpora. Enterprise procurement teams evaluate models against those figures. The number is real; the test set is incomplete.
The same dynamic surfaces in other AI domains. Cohere announced North Mini Code on 9 June 2026 — described by the company as its first model purpose-built for developers — precisely because general-purpose model scores conceal underperformance on domain-specific tasks. An aggregate accuracy figure passes procurement review. The production gap surfaces later.
IBM Research made the structural argument in a June 2026 analysis published on Hugging Face: according to that research, scalable enterprise AI adoption depends on agent logic and implementation-layer decisions, not on the frontier model selected at the top of the stack. A mismatched ASR layer is precisely this kind of implementation failure — invisible in headline benchmarks, consequential in production.
Three levers to close the gap
- Add a code-switching clause to every voice AI procurement RFP. Require vendors to provide benchmark results on multilingual, code-switched test sets before any contract is signed. ServiceNow AI's research published on 9 June 2026 provides a reference methodology — cite it explicitly in the specification.
- Run a bilingual stress test before go-live. Build a synthetic test set of ten to fifteen realistic bilingual exchanges covering your primary language pair. Run it against the ASR pipeline before any customer-facing deployment. One afternoon of testing avoids months of post-launch remediation.
- Add a language-detection layer upstream of ASR transcription. Explicit language identification, placed before the transcription step, allows the pipeline to route code-switched speech to a model benchmarked for that specific pair. This is an architectural choice independent of model selection — and it separates cleanly in any modular voice stack.
Is your voice pipeline ready for a bilingual customer?
If the honest answer is "the tests never covered that scenario," you now have a published benchmark framework to close that gap — and a structural argument for why it belongs in the next procurement cycle.
If this analysis speaks to you, I publish a piece of this calibre every day on digital innovation and enterprise AI. 👉 Get the next one straight in your inbox — sign-up takes ten seconds, and each edition is read before 9 a.m. by leaders of European SMEs, mid-caps and public institutions.