Meituan's LongCat-Video-Avatar 1.5 Beats HeyGen and Kling With Open-Source Weights
Meituan open-sources LongCat-Video-Avatar 1.5, a production-ready audio-driven avatar model that beats HeyGen and OmniHuman on human-likeness benchmarks under MIT license
Takeaways
- Meituan open-sources LongCat-Video-Avatar 1.5 under MIT license, enabling commercial use with no API fees.
- Replaces Wav2Vec2 with Whisper-Large-v3 (1.5B params, 680k hours of training data) for significantly better lip sync.
- Beats HeyGen, OmniHuman-1.5, and Kling Avatar 2.0 in pairwise human-likeness preference tests on a 508-sample benchmark.
- 8-step distilled inference via DMD2 with LoRA adapters; optional INT8 quantization reduces VRAM requirements.
- Supports multi-person conversations, anime/animal characters, and long-form video continuation with stable identity.
- Requires 2+ GPUs and manual setup; not available via Hugging Face Inference Providers yet.
Meituan's LongCat team just dropped something that should make commercial avatar vendors nervous. LongCat-Video-Avatar 1.5 is a fully open-source, MIT-licensed framework that takes a single portrait image and a speech audio clip and outputs a lip-synced, identity-stable video of that character speaking. No proprietary API, no subscription, no per-minute billing. Just weights you can run yourself.
The team explicitly prioritized systematic engineering and production-readiness over architectural novelty, achieving accurate lip-synchronization, full-body temporal stability, and robust long-video generation with strict identity consistency. That framing matters: this is not a research demo. Unlike many research demos that look amazing in cherry-picked clips but collapse in real usage, LongCat-Video-Avatar 1.5 is heavily optimized for stability, long-form generation, lip sync quality, and multi-character interactions.
One encoder swap, massive gains
The most consequential change in v1.5 is the audio encoder. Built on the 13.6B-parameter foundational video generation model LongCat-Video, the framework upgrades its audio encoder from Wav2Vec2 to Whisper-Large-v3 for significantly improved lip-sync accuracy. That is not a minor tweak. Whisper-Large has 1.5 billion parameters versus Wav2Vec2's 94 million, and was pre-trained on 680,000 hours of multilingual speech. The richer acoustic representations it produces let the model capture fine-grained phoneme-level dynamics that the older encoder simply missed.
The architecture injects audio into the video diffusion process through a dedicated cross-attention layer added to each DiT (Diffusion Transformer) block, placed right after the text cross-attention module. An Adaptive Layer Normalization gate controls how strongly audio influences each block, which prevents the model from catastrophically forgetting its visual priors while learning to sync lips. Audio features are extracted at 50 Hz, grouped into a 5-channel representation via mean pooling across Whisper's 33 hidden layers, then resampled to match the video's 25 FPS latent rate before injection.

Three-stage training pipeline
The model goes through three distinct training stages, each solving a different problem: