Skip to main content
Back

Alibaba's Qwen-Audio-Agent Hits 91% Success Splitting Voice Tasks Between Two AI Contexts

Alibaba's Qwen team splits a voice assistant into a fast talker and a slow thinker, hitting 91% task success on a cockpit benchmark.

Paper, Full Access
  • Qwen-Audio-Agent splits voice assistants into a fast Frontend Agent and slower Backend Agent coordinated by a runtime.
  • Runtime decouples speech interruption from task cancellation, so cutting off the assistant does not kill background work.
  • Mixed routing hits 91.04% task success on a 134-case cockpit benchmark, beating direct (72.39%) and all-delegated (80.60%).
  • Same mixed setup cuts mean task latency by 26.73% and 30.91% versus the two single-mode baselines.
  • Uses Qwen Audio 3.0 Realtime Plus as frontend and qwen3.8-max as backend, with 41 shared business tools.
  • Targets desktop assistance, intelligent cockpits, and voice customer service; no public code release accompanies the report.

Alibaba’s Qwen team has described Qwen-Audio-Agent, an architecture that keeps a voice conversation running while longer tasks execute in a separate model context. An orchestration runtime routes quick actions to the foreground model and delegates multi-step work to a background agent.

The technical report says mixed routing achieved 91.04% task success on a 134-case in-house cockpit benchmark. It also recorded lower latency on matched successful turns than configurations that routed every tool call through the foreground or background model.

One Conversation, Two Contexts

Speech-native models can stream dialogue and call tools, but complex workflows often consume conversational context and delay the next spoken turn. A request to plan a multi-stop route, reserve a table, and notify a passenger may require several tool calls, confirmations, retries, and state updates.

Qwen-Audio-Agent assigns those responsibilities across four components:

Core components in the Qwen-Audio-Agent architecture
Component Responsibilities
Frontend Agent Processes live audio, manages dialogue, handles direct tool calls, and decides when to delegate.
Backend Agent Executes delegated, multi-step tasks in a separate context and returns structured results.
Orchestration Runtime Tracks task state, coordinates authorization and user input, processes events, and schedules result delivery.
Client adapters Supply visual, application, and device state while reporting whether requested actions succeeded.

The frontend supports full-duplex interaction, meaning it can receive speech while generating audio. Separating the backend context also keeps long tool traces from filling the conversation history used for immediate responses.

The Runtime Protects In-Flight Work

The orchestration runtime maintains each task’s lifecycle while the frontend continues the conversation. It records pending work, routes requests for missing information, obtains authorization for sensitive actions, and holds completed results until the dialogue can accommodate them.