DeepSeek's DeepEP V2 Cuts GPU Communication Overhead From 24 to 4 SMs
DeepSeek open-sources DeepEP V2, a complete rewrite of its MoE communication library that cuts GPU SM usage by up to 4x while boosting throughput
Takeaways
- DeepEP V2 released: DeepSeek open-sources a complete rewrite of its MoE expert-parallel communication library with major efficiency gains.
- 4x fewer SMs: V3-like training workloads drop from 24 to 4-6 Streaming Multiprocessors while maintaining equivalent or better performance.
- New NCCL Gin backend: Replaces the NVSHMEM dependency with a lighter, header-only transport that reuses existing NCCL communicators.
- Unified ElasticBuffer API: High-throughput and low-latency dispatch/combine paths are merged into one interface with analytical SM tuning (no grid search).
- Scale up to EP2048: V2 supports significantly larger expert-parallel domains than V1, with 0-SM primitives for pipeline, context parallelism, and remote memory access.
- MIT licensed, free to use: Available now at github.com/deepseek-ai/DeepEP; requires Hopper (SM90) GPUs and NCCL 2.30.4+.
DeepEP, DeepSeek's open-source communication library for Mixture-of-Experts models, just shipped a major V2 rewrite. The headline number: the same expert-parallel communication workloads that consumed 24 Streaming Multiprocessors (SMs) in V1 now run on just 4 to 6, while delivering equivalent or better performance. For teams training or serving large MoE models at scale, that freed compute headroom is significant.
The problem DeepEP solves
In a Mixture-of-Experts model, each token gets routed to a small subset of specialized sub-networks ("experts") that may live on completely different GPUs. MoE models require two all-to-all exchanges per layer to dispatch tokens to experts and combine their outputs, and because transfer sizes and destinations are determined dynamically by the router at each step, standard collective communication proves inefficient since it is typically designed for fixed transfer sizes known ahead of time.
When the router assigns a token to an expert on a different GPU, all computation pauses while that token is shipped over NVLink via NCCL all-to-all. On an 8x H200 NVLink cluster, this communication block can consume 12 to 18% of per-step wall time depending on batch size and model depth. DeepEP was built to eliminate that stall.
What changed in V2
With the evolution of hardware, networking, and model architectures, the previous DeepEP V1 had accumulated too much legacy baggage and performance issues. V2 is a complete refactoring of Expert Parallelism, achieving extreme performance with several times fewer SM resources compared to V1, while supporting significantly larger scale-up and scale-out domains.
The three biggest architectural shifts are:
- New backend: V2 has switched from the NVSHMEM backend to the more lightweight NCCL Gin backend. NCCL Gin is a header-only, lightweight transport layer designed to provide high-performance RDMA and NVLink primitives while maintaining a minimal SM footprint. It allows DeepEP to reuse existing NCCL communicators and windows, simplifying deployment in standard distributed training environments.