Skip to main content
Back

Open-Source video-to-game Turns AI Clips Into Playable Browser Games for $6

An open-source agent skill turns a short AI-generated clip into a playable Three.js browser game, with every asset synthesized to match the video's look.

Repo, Full Access
  • Open-source skill video-to-game turns a 10-15s AI video into a playable Three.js browser game.
  • Works with Claude Code, Codex, Cursor, Gemini CLI and Copilot via the vercel-labs/skills standard.
  • Gallery of 31 games playable at fal-video-to-game.vercel.app, each a single index.html.
  • Total cost: $202 across 1,013 fal calls, averaging about $6 per finished game.
  • Pipeline studies the video, generates all images, 3D models and sounds on fal, then agent plays and records itself.
  • Requires Python 3.9+, ffmpeg, Node 18+, Playwright and a FAL_KEY; MIT licensed.

The open-source video-to-game repository gives a coding agent a 10 to 15-second AI-generated gameplay clip and instructs it to rebuild the scene as a playable browser game. The result uses Three.js, the JavaScript 3D library, and treats the clip as a visual specification for camera position, art direction, motion, and the core interaction.

The repository packages this workflow as a skill, a reusable instruction bundle compatible with Claude Code, Codex, Cursor, Gemini CLI, Copilot, and other coding agents. It has crossed 540 GitHub stars, and its live gallery contains 31 playable games.

Why the clip comes first

A text prompt such as “a boy glides between flying whales” leaves decisions about composition, materials, lighting, movement, and camera behavior unresolved. Coding agents often fill those gaps with generic geometry and default rendering choices. A short gameplay video supplies those missing decisions in a form the agent can inspect.

The video model first defines visible behavior and a coherent shot, and the coding agent then translates that reference into meshes, controls, lighting, effects, and browser logic. Repeated visual comparisons keep the implementation aligned with the source throughout development.

The loop, step by step

The skill runs a fixed sequence of analysis, generation, implementation, and verification tasks:

  1. Analyze the clip. study_video.py extracts frames, identifies scene cuts, builds a contact sheet, and derives a color palette. The contact sheet presents sampled frames in one grid for quick visual inspection.