Google's CUBE Beats Old 3D Face Models With 0.95mm Precision
Google researchers introduce CUBE, a hybrid B-spline and neural representation for 3D faces that beats prior scan registration methods while enabling local editing.
Takeaways
- Google introduces CUBE, a hybrid B-spline plus MLP representation for 3D faces (paper).
- Replaces 3D control points with lattices of high-dimensional learnable features, keeping B-spline local support.
- A transformer encoder predicts the control lattice from point clouds or images.
- Achieves 0.95mm point-to-scan error, beating BPS and TEMPEH on 18K real scans.
- Enables local editing, expression transfer, interpolation, and topology swaps without retraining.
- Trained on 300K synthetic scan-mesh pairs; inference 0.2-2.9s on V100.
A team at Google has proposed a new way to represent 3D human faces that borrows a decades-old idea from computer-aided design and gives it a neural upgrade. The representation, called CUBE (Control-based Unified B-spline Encoding), replaces the low-dimensional 3D control points of a traditional B-spline volume with high-dimensional learnable feature vectors, then bolts on a tiny MLP to add fine geometric detail. The result is a face model that supports precise local editing and, when paired with a transformer encoder, sets a new state of the art on facial scan registration.
The mesh problem nobody has quite solved
3D face pipelines for VR, telepresence, and avatars all hit the same tension. 3D Morphable Models built on triangle meshes have remained the default in academia and industry, but they lock you into a fixed topology and a low-dimensional parameter space that struggles with person-specific detail. Newer implicit approaches (signed distance fields decoded by MLPs) capture fine geometry but lack semantic correspondence between faces and require Marching Cubes to extract a mesh. B-spline surfaces from the CAD world offer a middle ground with smooth, editable control lattices, though their limited control points can't represent something as complex as a face.
CUBE tries to keep the useful bits from all three worlds: the semantic correspondence of a template mesh, the expressivity of learned features, and the localized editability of a B-spline.