Skip to main content
Back

VisionClaw Turns Meta Ray-Ban Glasses Into a Gemini-Powered Wearable Agent

An open-source project turns Meta Ray-Ban glasses into an always-on AI agent that sees, hears, and acts through Gemini Live plus OpenClaw.

Repo, Full Access
  • VisionClaw is an open-source real-time AI assistant for Meta Ray-Ban glasses using voice, vision, and agentic tool calls.
  • Pipeline streams glasses video at ~1fps and 16kHz audio to Gemini Live over WebSocket.
  • Optional OpenClaw gateway adds 56+ skills like messaging, web search, smart home, and lists.
  • Works on iOS 17+ and Android 14+; iPhone or phone camera mode lets you test without glasses.
  • Requires unlocking hidden Developer Mode in the Meta AI app by tapping app version five times.
  • Limitations: build-from-source only, WebRTC and Gemini Live cannot run simultaneously, OpenClaw browser-tab bugs.

A new open-source project called VisionClaw stitches together three moving parts that hobbyists have been trying to glue together on their own: Meta Ray-Ban smart glasses, Google's Gemini Live API, and a local agent gateway called OpenClaw. The result is a wearable assistant you can talk to, that sees what you see, and that can complete tasks on your phone or computer.

Put on the glasses, tap the AI button, and start talking. The camera streams to Gemini for visual context while audio flows bidirectionally in real time. Ask "What am I looking at?" and Gemini describes the scene through the glasses camera. Say "Add milk to my shopping list" and the request gets delegated to OpenClaw, which adds it through your connected apps. Messaging, web search, and smart-home actions all route through the same pipeline.

The pipeline under the hood

The architecture is refreshingly legible. Gemini Live provides real-time voice and vision over WebSocket with native audio rather than a speech-to-text-first pipeline. OpenClaw is an optional local gateway that gives Gemini access to 56+ tools and connected apps. A phone mode lets you test the full pipeline using your phone camera instead of glasses, and WebRTC streaming can share the glasses point of view live to a browser viewer.