Skip to main content
Back

DeepMind's Model Can Build Playable 3D environments

READ MORE

December 4, 2024 4 min read

Hey,

Welcome to AlphaSignal – the most read newsletter by AI developers. 


We bring you the top 1% of news, papers, models, and repos, all summarized to keep you updated on the latest in AI.

IN TODAY'S SIGNAL

Read time: 4 min 10 sec

🎖️ Top News


📌 Lambda


⚡️ Trending Signals

📌 Fiddler

⚙️ Python Tip

  • Context Managers: the secret weapon for clean ML experiments

🧠 Top Lecture

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

Language Models

DeepMind announces Genie 2: a foundation world model capable of generating an endless variety of playable 3D environments

⇧ 1255 Likes

What's New

DeepMind just announced Genie 2, a large-scale foundation world model that generates diverse, playable 3D environments for training embodied agents.


You can generate these environments from a single text or image prompt and control them with keyboard and mouse inputs. Worlds remain consistent for up to one minute.

How Genie 2 Works
Genie 2 uses an autoregressive latent diffusion model trained on a large-scale video dataset. It processes video frames through an autoencoder and generates new frames with a large transformer dynamics model that applies a causal mask.


You can provide an input prompt (e.g., a scene description) through Imagen 3, DeepMind’s text-to-image model, to create custom environments. Classifier-free guidance ensures your inputs control the correct elements in the scene.


Key Features

  • Generate 3D environments using image prompts or real-world photos.

  • Model interactions, including opening doors, moving objects, and applying realistic physics like gravity, water flow, and smoke.

  • Adapt environments to multiple perspectives, including first-person, isometric, and third-person views.

  • Simulate counterfactual scenarios by changing agent actions from the same starting frame.
  • Integrate and animate multiple agents with dynamic interactions.

Applications in AI Training
Researchers test agents with environments created by Genie 2. For example, using the SIMA agent, you can give natural-language instructions like “Open the red door,” and SIMA completes tasks by controlling an avatar in the environment.


Genie 2 supports long-horizon memory, enabling agents to explore environments consistently over extended interactions.

Read more

Get 8x better cost per token using NVIDIA GH200 on Lambda’s On-Demand Cloud

When running LLM inference, large models that don’t fit into a single GPU’s memory face two options: costly multi-GPU instances or performance hits with CPU offloading bottlenecks.


The NVIDIA GH200 Grace Hopper Superchip eliminates this tradeoff:

  • 7.6x higher throughput than H100 SXM for Llama 3 70B inference
  • 8x lower cost per token
  • Zero CPU offloading bottlenecks
  • Full model loading in unified memory

It is now available on-demand on Lambda’s Public Cloud so you can focus on shipping models.

Learn more

partner with us

TRENDING SIGNALS

OpenAI

Sam Altman announces the "12 Days of OpenAI" where each weekday the company will livestream a launch or demo

⇧ 11,047 Likes

Model Architecture

Liquid AI’s new STAR model architecture outshines Transformers

⇧ 410 Likes

Generative AI

Google opens access to Veo, a generative video model, and Imagen 3 on Vertex AI

⇧ 1601 Likes

Video Generation

Tencent releases HunyuanVideo, an open-source, open-weights, 13B parameter AI video generation model that beats top closed rivals

⇧ 581 Likes

Web

Exa opens the waitlist for an AI that can generate web-based datasets from any prompt

⇧ 1161 Likes

Webinar: How to meet compliance standards for LLM deployments

Fiddler AI and Credo AI discuss strategies to meet evolving LLM regulations and ensure enterprise growth.

  • Learn to meet GRC standards for Generative AI with evidence and documentation.
  • Mitigate hidden risks like bias and privacy in real time.
  • Prepare for regulatory changes with comprehensive audit trails.

All registrants receive access to the on-demand replay.

Signup ↗️

PYTHON TIP

Context Managers: The secret weapon for clean ML experiments

The @contextmanager decorator lets you create context managers for automatically handling experiment setup and cleanup. This is invaluable for ensuring consistent experiment environments and resource management.


This ensures reproducible experiments and automatic cleanup of GPU memory, regardless of errors. Perfect for research code where you're running multiple experiments and need consistent environments.


from contextlib import contextmanager
import torch

@contextmanager
def training_context(seed=42, device="cuda"):
    # Setup
    old_state = torch.get_rng_state()
    torch.manual_seed(seed)
    torch.cuda.empty_cache()
    
    try:
        yield # Your training code runs here
    finally:
        # Cleanup
        torch.set_rng_state(old_state)
        torch.cuda.empty_cache()
        
# Usage
with training_context(seed=123):
    model.train()
    # All code here runs with fixed seed
    # GPU memory auto-cleared after


TOP LECTURE

Small LLM

Hugging Face launches a free, hands-on course for aligning small LMs

⇧ 693 Likes

This hands-on course from Hugging Face teaches you how to customize and deploy small language models on your local machine. It shows you practical techniques for model alignment and fine-tuning without requiring expensive hardware.


You'll work with the SmolLM2 model series to learn skills that transfer to larger models. The playful name "smol" emphasizes its focus on small, efficient models that run on standard computers.

You will learn:

  • How to perform supervised fine-tuning and implement chat templates
  • DPO and ORPO techniques for aligning models with human preferences
  • Parameter-efficient methods like LoRA and prompt tuning
  • Creating and validating custom evaluation benchmarks
  • Adapting multimodal models for vision-language tasks
  • Building synthetic training datasets
  • Optimizing inference performance
  • Deploying models at scale while managing computational resources

The course needs basic Python, PyTorch, and transformers library knowledge. It runs on standard machines with minimal GPU requirements.

Read more

At Alpha Signal, our mission is to build a sharp, engaged community focused on AI, machine learning, and cutting-edge language models, helping over 200,000 developers stay informed and ahead. We’re passionate about curating the best in AI, from top research and trending technical blogs to expert insights and tailored job opportunities. We keep you connected to the breakthroughs and discussions that matter, so you can stay in the loop without endless searching. We also work closely with partners who value the future of AI, including employers and advertisers who want to reach an audience as passionate about AI as we are.


Our partnerships are based on shared values of ethics, responsibility, and a commitment to building a better world through technology.Privacy is a priority at Alpha Signal. Our Privacy Policy clearly explains how we collect, store, and use your personal and non-personal information. By using our website, you accept these terms, which you can review on our website. This policy applies across all Alpha Signal pages, outlining your rights and how to contact us if you want to adjust the use of your information. We’re based in the United States. By using our site, you agree to be governed by U.S. laws.

Looking to promote your company, product, service, or event to 200,000+ AI developers?

Work with us