CopilotKit's OpenGenerativeUI Turns AI Text Replies Into Live Interactive Visuals
CopilotKit released an open-source framework that lets AI agents stream live HTML, SVG, and 3D scenes into your app instead of just text.
Takeaways
- CopilotKit released OpenGenerativeUI, an MIT-licensed framework for agents that stream interactive HTML, SVG, and 3D visuals
- Built on LangChain Deep Agents with a skills-based architecture using on-demand SKILL.md files
- Rendering uses sandboxed iframes with Idiomorph DOM morphing for flicker-free streaming updates
- Ships a standalone MCP server compatible with Claude Desktop, Claude Code, and Cursor
- Requires strong models like Claude or GPT-class; smaller models produce broken layouts
- Live demo at opengenerativeui.copilotkit.ai covers 3D scenes, dashboards, algorithms, and diagrams
Chatbots that reply with walls of text feel dated when the underlying model can just as easily write a working SVG diagram or a Three.js scene. OpenGenerativeUI is CopilotKit's attempt to make interactive visuals the default: an MIT-licensed reference stack for building agents that respond with fully interactive components rendered live inside your app.
The project showcases rich AI-generated UI built on CopilotKit and LangChain Deep Agents. Ask it to visualize an algorithm, create a 3D animation, render a chart, or generate an interactive diagram, and the output comes back as live HTML/SVG inside a sandboxed iframe. The hosted demo ships prompts for a 3D solar system, a binary search walkthrough, a KPI dashboard, and a neural network with an animated forward pass.
Text is the wrong output for half of what agents get asked
The framing here is a decision matrix rather than a single trick. The agent produces interactive visual components across several categories: algorithm visualizations like binary search, BFS vs DFS, and sorting; 3D animations using WebGL and CSS3D; charts and diagrams including pie, bar, and network graphs; and interactive widgets like forms, simulations, and math plots. Everything renders in sandboxed iframes with automatic light/dark theming, progressive reveal animations, and responsive sizing.
Routing logic is explicit in the repo. Ask about a physical process, you get SVG. Ask about trends over time, you get Chart.js. Ask for a 3D visualization, you get Three.js. Ask for emotional support, you get warm text and no visual at all. That last row matters more than it looks, because it means the agent is trained to skip decorating every response with a chart.