ETH Zurich's SpaceControl Lets Rough 3D Sketches Guide AI Asset Generation Without Retraining
A training-free method injects 3D geometry directly into the latent space of pretrained generators, giving users a fidelity dial without any retraining.
Takeaways
- SpaceControl adds spatial 3D control to pretrained generators like Trellis without any fine-tuning.
- An SDEdit-style approach voxelizes and encodes user geometry, then partially noises and denoises the latent.
- A single tau_0 parameter trades geometric faithfulness against visual realism, sweet spot around 4-6.
- Chamfer Distance beats Spice-E, Spice-E-T and Coin3D across ShapeNet chairs, tables, and unseen Toys4K.
- Superquadrics, defined by only 11 parameters, act as compact sketching primitives for interactive editing.
- Code and an interactive editor are available at the project repo.
Text prompts are ambiguous and reference images are painful to manipulate, which makes controlling the exact shape of a generated 3D asset one of the more frustrating gaps in modern generative pipelines. A team from ETH Zurich, Stanford, Technion and NVIDIA is proposing a different route with SpaceControl, a training-free method that lets you feed rough 3D primitives or full meshes directly to a pretrained generator and get back a detailed, textured asset that actually follows your geometry.
The trick is that no fine-tuning is required. SpaceControl introduces explicit spatial control into modern generative frameworks for text- or image-conditioned 3D generation, such as Trellis or SAM 3D, by directly encoding user-specified geometry into the latent space as explicit guidance. That means the same base model that produces text-to-3D outputs can suddenly accept a stack of blocky primitives as a shape prior.

Why controllability keeps breaking
Existing spatially controlled 3D generators split into two camps, and both have real costs. Training-based methods extend generative models by fine-tuning them to accept particular forms of geometric inputs, for example voxel conditioning in LION or primitive- and mesh-based inputs in Spice-E, delivering explicit spatial control but requiring additional training that often reduces generalization. Guidance-based methods such as Latent-NeRF operate at inference time but typically incur substantial optimization overhead, and hybrid approaches like Coin3D still need fine-tuning plus test-time optimization on radiance fields.
In practice this means you either retrain a model per object class (and lose generality), or wait through long per-sample optimization loops. Neither works well when an artist wants to nudge a chair's backrest and see the result immediately.