NOIR Beats U-Net's 0.75 Dice Score by Treating Medical Scans as Math Functions
A new framework treats medical images as continuous functions and learns operators between them, delivering resolution-independent segmentation, translation, and synthesis.
Takeaways
- NOIR reframes medical imaging tasks as operator learning between continuous functions instead of pixel grids (arXiv).
- Encodes each scan as an Implicit Neural Representation with a shared network and a per-sample latent code.
- A small MLP-based neural operator maps input latents to output latents for segmentation, completion, translation, and synthesis.
- Holds Dice at 0.93-0.94 on Shenzhen X-rays from 32x32 to 200x200, while U-Net drops from 0.95 to 0.75.
- Empirically satisfies the epsilon-ReNO property with aliasing bounds of 4e-6 on input latents across resolutions.
- Code is available at Sidaty1/NOIR-io; current version still requires paired supervision.
Most medical imaging models quietly assume that a scan is a fixed grid of pixels or voxels. A new framework called NOIR (Neural Operator mapping for Implicit Representations) throws that assumption out and treats scans as continuous mathematical functions instead. The result is a single approach that handles segmentation, shape completion, image-to-image translation, and synthesis, and keeps working even when you feed it images at resolutions it never saw during training.
Why the pixel grid is a problem
In the prevailing deep learning framework, medical images are represented as pixels or voxels arranged on regular lattices and fed directly into neural networks. While this representation has proven highly effective, it raises a fundamental question: is the discrete grid the most appropriate representation for medical images? The issue is that medical images originate from inherently continuous physical domains but are frequently resampled, interpolated, and transformed during clinical workflows, making discrete grid-based neural network models sensitive to voxel spacing, interpolation, and resampling artifacts.
In practice, a U-Net trained on 200x200 chest X-rays can fall apart when given a 32x32 crop, and a model tuned to 1mm voxels can misbehave on 2mm voxels. NOIR reframes the entire pipeline so that discretization becomes a rendering detail rather than a fundamental input format.
Functions in, functions out
NOIR embeds each medical signal into an Implicit Neural Representation (INR): a small coordinate-based neural network that maps a spatial location (x, y) or (x, y, z) to an intensity or class probability. Instead of training one INR per image, the authors share network weights across the whole dataset and give each sample a small latent code that modulates those weights, following the auto-decoder recipe used in recent INR work.