Skip to main content
Back

RadixArk's Miles Stops Frontier RL Training Runs From Silently Collapsing

RadixArk open-sources Miles, an enterprise RL framework built on slime that tackles MoE training collapse, FP8 stability, and rollout throughput at frontier scale.

Repo, Full Access
  • RadixArk open-sourced Miles, an enterprise RL framework forked from slime for LLM and VLM post-training.
  • Unified BF16/FP8/MXFP8/NVFP4/INT4-QAT pipeline eliminates training-inference mismatch that causes MoE RL collapse.
  • Rollout Routing Replay (R3) records SGLang expert routing and replays it during training for bitwise alignment.
  • Speculative rollout with online SFT draft model delivers 25%+ throughput gains versus frozen draft.
  • Supports GRPO, GSPO, PPO, REINFORCE++ across DeepSeek, Qwen, Llama, GLM, MiniMax and agentic environments.
  • Built on SGLang, Megatron-LM, Ray and PyTorch; available via Docker or pip.

Reinforcement learning on frontier-scale language models has a dirty secret: the algorithms look clean on paper, but training runs collapse in production because the numerics between the training engine and the inference engine drift apart. A new open-source framework called Miles from RadixArk is going after that problem directly, and it is arriving with real production users behind it.

Miles is an enterprise-facing reinforcement learning framework for LLM and VLM post-training, forked from and co-evolving with slime. The project was announced by RadixArk, a team spun up by core developers in SGLang and its extended ecosystem (slime, AReaL). It has already collected over 2,000 GitHub stars and, according to the launch thread, powers RL workloads at Modal, Decagon, Nebius, IBM, Periodic Labs, and others on both NVIDIA and AMD hardware.

The mismatch that eats MoE training runs

The core pitch is about correctness under scale. When you sample rollouts from an inference engine and then compute gradients in a training engine, tiny numeric differences (kernel choices, quantization paths, expert routing in mixture-of-experts models) create off-policy bias that can silently destroy a run. Beyond deterministic inference with bitwise identical results, Miles supports true on-policy training via an infrastructure approach, eliminating the mismatch between training and inference to exactly 0 KL divergence using Flash Attention 3, DeepGEMM, batch invariant kernels from Thinking Machines Lab, and torch.compile, and aligning numeric operations between training and inference.

When exact alignment is not achievable, Miles falls back to algorithmic fixes: Truncated Importance Sampling and Masked Importance Sampling to keep gradient estimates unbiased despite off-policy drift.