Skip to main content
Back

NVIDIA's CUTLASS 4.5 Ships Python GPU Kernels 1.41x Faster for AI Inference

CUTLASS 4.5 ships a simpler Python copy API, MXF8F6F4 mixed precision, and new MoE kernels with up to 1.41x speedup on Blackwell

Repo, Full Access
  • CUTLASS 4.5.x released: NVIDIA ships versions 4.5.0–4.5.2 of its GPU linear algebra library with Python DSL and C++ improvements.
  • New block_copy() API: Dramatically simplifies TMA and shared-to-tensor memory copy in CuTe DSL, removing significant boilerplate.
  • MoE GEMM speedups on B200: New grouped-GEMM examples show up to 1.41x speedup over PyTorch for mxfp8 workloads on Blackwell.
  • MXF8F6F4 mixed precision: BlockScaled MMA now supports MXF8*MXF4 and MXF8*MXF6 combinations for Blackwell kernels.
  • C++ gains: Up to 30% perf improvement on SM121 via new tile sizes; paged KV cache for Blackwell GQA; green context SM partitioning.
  • Install via pip: pip install nvidia-cutlass-dsl; free and open-source under BSD-3-Clause. Windows builds remain broken.

NVIDIA CUTLASS has shipped its 4.5.x release series, capping off with version 4.5.2. The headline additions span both sides of the library: the Python-native CuTe DSL gets a simpler memory-copy API and broader mixed-precision support, while the C++ side gains new Blackwell-specific tile sizes, green context partitioning, and paged KV cache support for attention kernels. For anyone writing custom GPU kernels for inference or training, there is a lot to unpack.

What CUTLASS Actually Is

CUTLASS is NVIDIA's open-source library of CUDA building blocks for high-performance linear algebra. Think of it as the layer between raw PTX assembly and framework-level operators like torch.matmul. CUTLASS decomposes matrix multiplication and related computations into reusable, modular software components and abstractions, with primitives that can be specialized via custom tiling sizes, data types, and algorithmic policy. It has been the engine behind cuBLAS and many deep learning framework kernels since 2017.

CUTLASS 4.0 introduced something new on top of the C++ template library: CuTe DSL, a Python-native interface for writing those same high-performance kernels. CUTLASS 4.x provides Python-native interfaces for writing high-performance CUDA kernels based on core CUTLASS and CuTe concepts without any performance compromises, allowing for a much smoother learning curve, orders of magnitude faster compile times, and native integration with DL frameworks without writing glue code. Under the hood, CUTLASS DSLs translate Python code into a custom intermediate representation (IR), which is then Just-In-Time (JIT) compiled into optimized CUDA kernels using MLIR and ptxas.

CUTLASS GEMM decomposition hierarchy showing tiling levels from thread block to warp to thread

The 4.5 Release in Plain English

The 4.5.0 drop is the most substantive of the three patch releases. Here is what changed in CuTe DSL: