Skip to main content
Back

AI4AnimationPy Drops Unity so Researchers Can Run Character AI in Pure Python

A landmark character animation research framework is leaving Unity behind, with a Python rewrite that unifies training, inference, and visualization.

Repo, Full Access
  • AI4Animation released AI4AnimationPy, a full Python rewrite dropping Unity as a dependency.
  • Training, inference, visualization, and data processing now run in one NumPy/PyTorch environment.
  • Demos cover biped locomotion, quadruped control, IK, motion capture import, and a motion editor.
  • Browser demos live at paulstarke-ai4animationpy.hf.space, no install required.
  • The SIGGRAPH 2024 codebook matching technique handles ambiguity in sparse-to-full-body mapping.
  • Licensed CC BY-NC 4.0, so research and education only, not commercial use.

Sebastian Starke's AI4Animation has been one of the most influential open research codebases in character animation for nearly a decade, powering a string of SIGGRAPH papers on neural locomotion, quadruped control, scene interaction, and VR avatars. The repository just crossed 8.8k stars and shipped its biggest structural change yet: a full Python remake, released as AI4AnimationPy, that drops Unity as a hard dependency.

Unity is out, PyTorch is in

The original AI4Animation was a hybrid beast. Data processing, feature extraction, and runtime inference lived in Unity/C#, while training happened separately in PyTorch, with data shuttled between the two. The new version collapses that split. This version brings AI4Animation to Python, removing the Unity dependency for data-processing, feature-extraction, inference, and post-processing while keeping a similar game-engine-style architecture including ECS, update loops, and a rendering pipeline. Everything now runs on NumPy or PyTorch inside a single environment.

Diagram comparing the old Unity-based pipeline to the unified Python architecture

For anyone who has tried to reproduce animation research, the pain point being solved here is obvious: you no longer need to install a specific Unity version, wire up custom C# inspectors, and hop between IDEs to iterate on a network. The framework uses RayLib for visualization and rolls its own ECS-style entity hierarchy so the runtime feel of a game engine stays intact.