Skip to main content
Back

Tencent's AuK Merges 16 Speech Tasks Into One Open-Source Model

Tencent open-sourced AuK and AuK-Flash, a 1.5B unified speech model that handles TTS, editing, enhancement, and separation from plain-language instructions.

  • Tencent open-sourced AuK, a 1.5B unified speech generation and editing model under MIT license.
  • AuK-Flash is a distilled 4-step variant offering 4.5x wall-clock speedup with no CFG.
  • One natural-language interface handles TTS, voice cloning, content editing, accent removal, denoising, and source separation.
  • Built on Qwen2.5-Omni-3B encoder, joint speech-audio-music VAE, and hybrid MMDiT plus DiT rectified-flow transformer.
  • Trained on ~3.03B instruction-audio pairs across 1.95M hours spanning five task families.
  • Ships with GitHub code, ComfyUI nodes, vLLM-Omni serving, MLX support, and a browser demo Space.

Tencent has released AuK and the distilled AuK-Flash on Hugging Face under an MIT license. The 1.5-billion-parameter generator handles text-to-speech, voice cloning, audio editing, enhancement, and source separation through a shared instruction interface. Developers can describe a task in English or Chinese and attach reference audio when the operation requires it.

Component Details
Generator 1.5 billion parameters
Conditioning encoder Qwen2.5-Omni-3B
Training data About 3.03 billion instruction-audio pairs covering 1.95 million hours
Task coverage 16 capabilities across five task families
Flash inference Four sampling steps without classifier-free guidance
License MIT

Many speech jobs, one interface

The project groups generation and editing capabilities that commonly require separate models, preprocessing pipelines, and audio formats. Zero-shot cloning uses a short reference recording to synthesize new speech in the same voice, while instruction-based editing modifies an existing recording according to a text prompt.

Supported operations include:

  • Generating speech from text with or without a reference voice
  • Replacing spoken words while preserving surrounding audio
  • Changing timbre, tone, emotion, or speaking style
  • Reducing a regional accent while retaining speaker identity and content
  • Adding coughs, laughter, breaths, and other nonverbal sounds
  • Removing noise and improving recording quality
  • Extracting a speaker from a mixed recording
  • Separating music stems
  • Converting whispered speech into normal voicing

A shared interface can reduce the glue code required for workflows that combine several operations. A pipeline for editing dialogue, removing background noise, and inserting a timed sound can send those requests to one model instead of coordinating multiple task-specific repositories.