NVIDIA's MatX Hits v1.0 and Runs 2100x Faster Than NumPy
NVIDIA's MatX hits v1.0.0 with CUDA JIT kernel fusion, bringing NumPy-style C++ tensor code to near-native GPU performance
Takeaways
- v1.0.0 released: NVIDIA's MatX hits its first stable release, now requiring C++20 and CUDA 12.2.1+.
- CUDA JIT kernel fusion: New
CUDAJitExecutorcompiles and caches fused kernels at runtime, converting runtime parameters to compile-time for faster execution. - FFT + GEMM fusion via MathDx: MatX can now fuse FFT and BLAS operations into larger arithmetic expressions using NVIDIA's MathDx libraries.
- 2100x faster than NumPy, 4x faster than CuPy: Benchmarked on an FFT resampler running on an A100 GPU vs. a Xeon CPU.
- Header-only, BSD-3 licensed: Drop into any CMake project; supports Volta through Blackwell GPUs and CPU execution with multithreading.
- New primitives: SAR backprojection transform, experimental sparse tensor type, and cuBLASDx integration added in this release.
MatX is NVIDIA's open-source C++20 library for GPU-accelerated numerical computing, and it just hit its first stable release: v1.0.0. The pitch is simple but powerful: write tensor math that looks like NumPy or MATLAB, and get performance that blows both out of the water. On a benchmark FFT resampler, the MatX version runs in 2.54ms on an A100 versus 5360ms for NumPy on a Xeon CPU and 10.6ms for CuPy on the same A100.
That's not a typo. The MatX version shows a 2100x speedup over NumPy, and over 4x faster than CuPy on the same GPU. The code to achieve this is roughly the same length and complexity as the Python equivalent, which is the whole point.
The Gap MatX Is Closing
The typical workflow for GPU numerical computing has always involved a painful tradeoff: write high-level Python (NumPy, SciPy) and leave massive performance on the table, or drop down to raw CUDA and spend weeks writing and debugging kernels. CuPy closes part of that gap by running NumPy-style code on the GPU, but it still carries Python overhead and can't match hand-tuned C++.
MatX is a modern C++ library for numerical computing on NVIDIA GPUs and CPUs, with compile-time expression evaluation for generating GPU kernels and near-native performance while using a syntax similar to Python or MATLAB. The goal is to make the C++ version feel like a direct translation of the Python version, not a rewrite.
How It Actually Works
The core trick behind MatX's performance is lazy evaluation combined with expression templates. When you write an expression like A = B * cos(C) / D, MatX doesn't immediately execute each operation.
Instead, it uses lazy evaluation to reduce the total number of memory loads and stores, by overloading each operator so that instead of performing the operation, it returns an object representing that operation when it's needed.