Skip to main content
Back

Brown and MIT's Steklov-CLIP Beats Laplacian Methods on 450,000 Messy 3D Shapes

Researchers from Brown and MIT introduce a Monte Carlo method for volumetric 3D shape analysis that runs orders of magnitude faster than prior approaches, scaling to 450,000 real-world meshes

  • Researchers introduce a Monte Carlo estimator for the Dirichlet-to-Neumann (Steklov) operator, orders of magnitude faster than prior boundary-element methods.
  • The method is robust to poor triangulations, self-intersections, and meshes with hundreds of disconnected components, unlike standard intrinsic operators.
  • An exterior DtN variant couples disconnected mesh components through ambient space using a Kelvin transform to handle the unbounded domain.
  • Steklov eigenspectra were computed for ~450,000 uncurated Objaverse shapes, the first volumetric spectral analysis at this scale.
  • Steklov-CLIP, a mesh-based CLIP model built on these operators, achieves 49.1% top-1 on Objaverse-LVIS zero-shot classification and 60.1% top-1 on part-caption retrieval after finetuning.
  • Code is available at github.com/mc-steklov/mc-steklov.

Geometry processing has long relied on the Laplacian operator as its workhorse. It is elegant, fast, and underpins everything from shape correspondence to mesh deformation. But it has a fundamental blind spot: it only sees the surface, not the volume. Two shapes that are isometrically equivalent (think a human in different poses) look identical to the Laplacian. And when you feed it the kind of messy, multi-component, poorly triangulated geometry that comes from the real world, it often breaks entirely.

A team from Brown University and MIT has now published a paper introducing a Monte Carlo method for estimating the Dirichlet-to-Neumann (DtN) operator, a volumetric boundary operator whose eigenmodes (called Steklov modes) encode the full 3D geometry of a shape. The key result: their method is orders of magnitude faster than prior approaches, robust to degenerate meshes, and scales to datasets of nearly half a million shapes. They also release the code at github.com/mc-steklov/mc-steklov.

The problem with intrinsic operators

The standard toolkit for mesh-based geometry processing is built on intrinsic operators, meaning operators that depend only on the surface metric, not the 3D embedding. The Laplace-Beltrami operator is the canonical example. Because it is invariant to isometric deformation, it is used everywhere: shape descriptors, functional maps, spectral neural networks like DiffusionNet.

But this invariance is also a limitation. Two shapes with the same metric but different 3D embeddings are indistinguishable. More practically, intrinsic methods assume:

  • Clean, manifold, single-component meshes
  • Reasonable triangle quality (no degenerate slivers)
  • Consistent surface orientation

Real-world 3D data, especially from sources like the Objaverse dataset, satisfies none of these. A Commodore 64 keyboard mesh, for instance, has 2,662 connected components. The cotangent Laplacian simply cannot couple them.

The Steklov spectrum: volumetric geometry on the boundary

The Dirichlet-to-Neumann (DtN) operator is a boundary-to-boundary map: given a scalar function on the surface, it returns the normal derivative of the harmonic extension of that function through the interior volume. Intuitively, it asks: if you set boundary values and let them diffuse through the interior, how much flux comes back out at each point?