Skip to main content
Back

NVIDIA's cuTile Python Rewrites GPU Kernels Without a Line of C++

NVIDIA's cuTile Python brings tile-based GPU kernel programming to Python, eliminating the need for C++ and manual thread management in CUDA 13.1

Repo, Full Access
  • New programming model: NVIDIA's cuTile Python ships with CUDA 13.1, letting you write GPU kernels in Python without any C++ or thread-level CUDA code.
  • Tile abstraction: Instead of managing individual threads, you operate on data tiles; the compiler handles thread orchestration, shared memory, and synchronization automatically.
  • Hardware support: Fully supported on Blackwell (B200/B300) and Ampere/Ada (A100, L40S) with CUDA 13.2; Hopper (H100/H200) is not yet supported.
  • Performance: Matches cuBLAS within 3% on A100 FP16 GEMM; approaches FlashAttention 3 throughput for attention kernels; exceeds cuBLAS on some Blackwell FP8 configs.
  • Free and open source: Available on PyPI as cuda-tile under Apache 2.0; install with pip install cuda-tile[tileiras].
  • Ecosystem context: Competes with Triton and JAX/Pallas but compiles through the official CUDA toolchain, giving it a direct path to future NVIDIA hardware optimizations.

Writing a custom GPU kernel has always meant wrestling with CUDA C++: thread indices, shared memory layouts, warp synchronization, and tensor core intrinsics that change with every GPU generation. cuTile Python is NVIDIA's answer to that friction. Shipped as part of CUDA 13.1, it introduces a tile-based programming model that lets you write high-performance GPU kernels entirely in Python, with the compiler handling the thread orchestration under the hood.

The biggest CUDA shift in 20 years

CUDA 13.1 shipped with something that hasn't happened in CUDA's 20-year history: a complete new programming model layered on top of the existing one. The release introduces tile-based programming for GPUs, making it one of the most fundamental additions to GPU programming since CUDA was invented. The core idea is a shift in abstraction level: instead of thinking in threads, you think in tiles.

In this model, arrays are the primary data structure. Tiles are subsets of arrays that kernels operate on. Kernels are functions executed in parallel by blocks. Blocks are subsets of the GPU, and operations on tiles are parallelized across each block. The compiler and runtime then figure out how to map all of that onto actual hardware threads.

Why thread-level CUDA is painful

Traditional CUDA programming relies on the SIMT (Single Instruction, Multiple Threads) model, where developers manually manage thread indices, blocks, and grids. While powerful, this low-level approach becomes increasingly complex as GPUs evolve with specialized hardware like tensor cores.

Writing code that performs well, especially across multiple GPU architectures, requires considerable effort. With more complex hardware, more software is needed to help harness these capabilities. CUDA Tile abstracts away tensor cores and their programming models so that code using CUDA Tile is compatible with current and future tensor core architectures.