Skip to main content
Back

Google Ships Gemini 3.5 Live Translate to Kill the 3-Model Pipeline

Google's new audio model streams speech translation across 70+ languages in near real-time, collapsing the classic transcribe-translate-synthesize pipeline into a single model call

  • Single-model architecture: Gemini 3.5 Live Translate collapses the classic speech-to-text → translate → text-to-speech pipeline into one audio model, reducing latency and error accumulation.
  • 70+ languages, 2,000+ combinations: Auto-detects source language and preserves speaker intonation, pacing, and pitch in the translated output.
  • Three simultaneous launches: Public preview on the Gemini Live API and AI Studio, Google Translate app globally, and Google Meet private preview for Workspace customers.
  • ~3-second latency: Independently measured median first-audio latency of 2,947 ms — consistent with Google's "a few seconds behind" claim.
  • $0.023/minute API pricing: Undercuts competing services, making it viable for consumer-scale audio apps.
  • Known limits: Voice consistency issues in multi-speaker sessions, struggles with non-native accents and rapid language switches, still in preview with no production SLA.

Google just shipped Gemini 3.5 Live Translate, a dedicated audio model that translates spoken language into spoken language as the audio streams in. No waiting for a sentence to finish. No discrete request-response cycle. Just continuous speech in, continuous translated speech out, staying a few seconds behind the speaker throughout the session.

The pipeline problem it kills

Every real-time translation system before this worked roughly the same way: capture speech, run it through a speech-to-text model, feed the transcript to a translation model, then synthesize the result with a text-to-speech engine. Three separate models, three separate latency budgets, three places for errors to compound.

Three architectural choices distinguish Gemini 3.5 Live Translate from earlier streaming-translation systems. Traditional pipelines run audio through a streaming speech-to-text model, feed the transcript to a machine-translation model, then synthesize the translation through a separate text-to-speech model. Each stage adds latency and accumulates errors. Gemini 3.5 Live Translate folds these steps into one audio model. The trade-off is real though: the output is permanent audio, not editable text , once a word is spoken, it cannot be revised mid-utterance.

Gemini is a natively multimodal model, meaning it was trained on text, images, code, and audio simultaneously rather than being a language model with audio features bolted on. This matters for translation because speech carries information that text doesn't , emphasis, hesitation, pace, and tone. Gemini 3.5 Live Translate is based on Gemini 3 Pro.

What it actually does

The model automatically detects 70+ languages and generates smooth, natural-sounding translated speech that preserves the speakers' intonation, pacing and pitch. Unlike turn-by-turn systems that wait for the speaker to finish speaking before responding, 3.5 Live Translate generates speech continuously, balancing the trade-off between waiting for context to improve quality and translating immediately to stay in sync with the speaker. It delivers fluid audio without awkward pauses and stays just a few seconds behind the speaker throughout the session.

Independent latency measurements back that up. First-audio latency clocks in at a median of 2,947 ms from start of speech to first translated audio (p10–p90: 2,859–3,104 ms), consistent with Google's "a few seconds behind" framing.

Key capabilities at a glance:

  • Covers more than 70 languages and 2,000+ language combinations.
  • Preserves intonation, tempo, and vocal pitch , so the translated voice sounds closer to how the original speaker actually sounds, rather than flat synthesized audio.
  • Handles multilingual inputs without the need to manually configure settings.
  • Handles background noise and overlapping voices, making it usable in real-world settings like busy cafés or open offices.
  • Outputs both translated audio and a text transcription in a single model call.

Known limitations

Google's own model card is refreshingly candid about the rough edges:

  • Voices can be inconsistent, and voices may shift after long pauses, change gender, or get stuck on one voice during rapid multi-speaker sessions.
  • Language detection can struggle with non-native accents, similar languages, or rapid language switches.
  • The model is designed to filter out background noise, but not all background audio may be ignored. When set to echo the target language, background noise may introduce artifacts in the translated audio when input audio is in the target language.
  • The model handles overlapping speech reasonably well in small groups, but accuracy can drop in large meetings where multiple people speak simultaneously. For best results in multilingual panel discussions, a structured format where speakers take clear turns produces more accurate translation output than a free-flowing conversation.

Worth noting: the latency, voice-preservation, and partner-scale claims are Google-stated and not yet independently benchmarked, and the model is in preview , no production SLA, no stable pricing, no GA.

Where it ships

The model is rolling out simultaneously to developers via the Gemini Live API, to enterprise customers in Google Meet, and to consumers through the Google Translate app on Android and iOS , marking one of the broadest single-day launches Google has staged for a translation product.

  • Developers: Access the model via the Gemini Live API and Google AI Studio, currently in public preview. The model identifier is gemini-3.5-live-translate-preview.
  • Google Meet: The Meet update is in private preview for select business Workspace customers this month, with a broader rollout following later this year. The upgrade is significant: language support expands from five to more than 70, and the system will support conversations across more than 2,000 language combinations in a single meeting, compared to the previous English-only constraint.
  • Google Translate app: The consumer app update is live globally on Android and iOS , no sign-up required. Android users also get a new Listening Mode: hold the phone to your ear like a normal call and hear a private translation through the speaker , no earbuds needed.

Calling it from the API

Streaming audio in and getting translated audio plus transcription back is a single call to the Gemini Live API. The model handles language detection automatically, so you don't need to specify source or target languages upfront. Here's the minimal pattern:

import asyncio
from google import genai
client = genai.Client()
async def translate_stream():
    async with client.aio.live.connect(
        model="gemini-3.5-live-translate-preview"
    ) as session:
        # Stream raw audio bytes in
        await session.send_audio(audio_bytes)
        # Receive translated audio + text transcript
        async for response in session.receive():
            if response.audio:
                play(response.audio)
            if response.text:
                print(response.text)

Developer platforms like Agora, Fishjam, LiveKit, Pipecat, and Vision Agents already integrate with the Gemini Live API and handle the complex real-time media streaming infrastructure, so developers can focus on the user experience.

Pricing and early adoption

At $0.023 per minute with 70+ languages, Gemini 3.5 Live Translate undercuts OpenAI on price. That's roughly $1.38 per hour of translated audio , a number that makes it viable to embed in consumer-facing apps at scale.

Grab is testing the model to enable multilingual communication in near real-time between drivers and travelers at pickups , a user base that makes over 10 million voice calls per month through the platform. Companies like CJ ENM, LiveKit and others have shared positive feedback highlighting impressive translation quality, accuracy and low latency.

Why the architecture shift matters

The jump from text-based translation to continuous speech-to-speech translation , preserving voice, intonation, and pitch in real time , represents a qualitative shift in what machine translation can do. Text translation operates on completed utterances. Speech-to-speech translation at this capability level requires the model to make decisions about how to render language while the speaker is still mid-sentence, without waiting for semantic closure.

The practical use cases this unlocks:

  • Live multilingual customer support calls without human interpreters
  • Real-time dubbing for live video broadcasts
  • Ride-share and travel apps bridging driver-passenger language gaps
  • International business meetings without language constraints
  • Educational platforms delivering lectures across language barriers

One safety note worth flagging: all audio generated by the model is watermarked with SynthID , an imperceptible watermark woven directly into the audio output, ensuring AI-generated content remains detectable to help prevent misinformation.

The field assumption that needs updating is this: real-time speech translation was previously treated as a pipeline engineering problem , how fast can you chain three models together? Gemini 3.5 Live Translate reframes it as a single-model audio understanding problem. That's a different design space entirely, and it's one that gets better as the underlying model improves, not just as the infrastructure gets faster.