Baidu's PaddleFormers Ships GLM-4.5 MTP Training With 48% VLM Speed Boost
Baidu's PaddleFormers v1.1 adds MTP training for GLM-4.5 and a 48% VLM performance gain over its previous release
Takeaways
- PaddleFormers v1.1 released by Baidu's PaddlePaddle team, adding MTP training support for GLM-4.5 and major VLM performance gains.
- Multi-token prediction (MTP) training for GLM-4.5 now supported, enabling both better training signal and faster speculative decoding at inference.
- 48% VLM performance improvement for Qwen3-VL 30B-A3B, now outperforming Megatron-LM by 6% on training throughput.
- 100+ models supported including DeepSeek-V3, Qwen3, GLM-4.5, Llama-3, and Baidu's own ERNIE-4.5 series.
- Hugging Face-compatible API and checkpoint format (Safetensors); trained weights deploy directly to vLLM, SGLang, and FastDeploy.
- Free and open-source under Apache 2.0; available at github.com/PaddlePaddle/PaddleFormers.
PaddleFormers, Baidu's open-source LLM training library built on the PaddlePaddle framework, has shipped its v1.1 release. The update lands a few months after the landmark v1.0 milestone and focuses on two concrete improvements: multi-token prediction (MTP) training support for the GLM-4.5 model family, and a significant performance jump for vision-language model (VLM) training.
What PaddleFormers actually is
PaddleFormers is a full-stack large model training toolkit built on PaddlePaddle, engineered to streamline adaptation, boost performance, and enable deployment across diverse hardware environments. Think of it as Baidu's answer to Hugging Face Transformers, but with the training infrastructure dialed up. It covers end-to-end training from pre-training to post-training for LLMs and VLMs, supporting 100+ mainstream large language and vision-language models, including ERNIE-4.5, DeepSeek-V3, the GLM-4.5 series, and Qwen2/3 series.
The API is intentionally familiar. If you've used Hugging Face Transformers, the surface looks almost identical:
from paddleformers.transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B-Base")
model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B-Base", dtype="bfloat16").eval()
input_features = tokenizer("Explain transformers briefly:", return_tensors="pd")
outputs = model.generate(**input_features, max_new_tokens=256)