Skip to main content
Back

NVIDIA Launches Cosmos for Faster Synthetic Data Generation

READ MORE

January 8, 2025 4 min read

Hey,

Welcome to AlphaSignal – the most read newsletter by AI developers. 


We bring you the top 1% of news, papers, models, and repos, all summarized to keep you updated on the latest in AI.

IN TODAY'S SIGNAL

Read time: 4 min 44 sec

🎖️ Top News

📌 Lambda

⚡️ Trending Signals

📌 DataCrunch

💻 Top Models

🧠 PyTorch Tip

  • Use PyTorch's fault tolerance for uninterrupted distributed training in large-scale models.

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

Generative AI

NVIDIA launches Cosmos with open-source video models to generate synthetic data for robotics

⇧ 8,498 Likes

What's New

NVIDIA Cosmos introduces open-source, open-weight Video World Models to support physical AI development, including autonomous vehicles and robotics. It addresses data scarcity in robotics by generating synthetic datasets at scale. Trained on 20 million hours of video data and 9,000 trillion tokens, Cosmos provides tools for creating physics-aware video simulations for various applications.

Model Architecture

  • Contains two types of model architectures.
  • Diffusion-based models: Generate continuous tokens to create controllable visual simulations.

  • Autoregressive models: Use discrete tokens to predict future video frames.

  • Support text-to-video generation: Simulate scenarios from text prompts.

  • Support video-to-video generation: Create simulations from video inputs.

  • Enable combined inputs: Use both text and video to build realistic simulations.

Model Variants for Specific Needs
Cosmos offers three model variants:

  • Nano: Optimized for low-latency, real-time inference and edge deployment.

  • Super: Baseline models for general performance.

  • Ultra: High-quality, high-fidelity models suitable for fine-tuning custom models.

Pre-Trained Models and Developer Tools
You can access pre-trained models, fine-tuning scripts via NVIDIA NeMo, and tools like video tokenizers and a video curation pipeline. Training scripts and post-training resources allow users to customize models for domain-specific tasks. Pre-trained models include vertical-specific fine-tunes, such as multisensor data generation for autonomous vehicles.

Availability and Licensing
Cosmos is available on Hugging Face and the NGC catalog under the NVIDIA Open Model License, permitting free commercial use. Training scripts are licensed under Apache 2, and preview access is offered at build.nvidia.com.

Read more

Future-Proof Your Workloads with NVIDIA GH200s on Lambda

NVIDIA Blackwell is almost here, bringing ARM-based computing. The NVIDIA GB200 Superchip pairs two B200 Tensor Core GPUs with the Grace CPU, connected via a 900GB/s NVLink interconnect.

But if your workflows and tools are designed for x86, they need testing and possibly recompiling to run properly on Grace Blackwell.

Test your workflows now on the NVIDIA GH200s (ARM64 CPU + Tensor Core GPU) available on Lambda Cloud. The GH200s offers 3x the performance of the A100 at a similar cost, giving you a solid preview of what to expect from Blackwell.

Don’t wait. Test your workloads now to ensure they’re ready for the ARM-based future.

Learn more

partner with us →

TRENDING SIGNALS

AI Hardware

NVIDIA collaborates with NVIDIA to enable Llama fine-tuning on desktops with NVIDIA's new personal AI supercomputer, DIGITS

⇧ 3,635 Likes

Open Source LLM

Together AI launches DeepSeek-V3 with 131k context and opt-out privacy for scalable AI apps, featuring fine-tuning and no rate limits 

⇧ 725 Likes

Agent

LangChain unveils structured report generation agent with optimized Llama 3.3 and LangGraph integration

⇧ 834 Likes

ML Optimization

Meta updates QINCo2 repository with optimized neural codebooks for residual quantization which measures inductive bias in models

⇧ 592 Likes

LLM

US-based startup releases a series of open-source, local-first AI models offering full control over alignment and deployment options

⇧ 563 Likes

Get access to on-demand GPU clusters designed for AI workloads.

Accelerate AI training and inference with custom GPU clusters from DataCrunch.

  • Support from experienced AI engineers

  • Compliance with EU regulations

  • Powered by renewable energy

Ensure uptime with SLA-backed support, low-cost high-throughput storage, and flexible contracts.

Start now ↗️

TOP MODELS

Generative AI

Cosmos-1.0-Diffusion-7B-Text2World

⇧54 Downloads

Cosmos helps you build and fine-tune world models for Physical AI systems, focusing on robotics and AV labs. It provides pre-trained Diffusion and Autoregressive models for Text2World and Video2World generation, training scripts, video tokenizers, and a curation pipeline. Models are available on Hugging Face for commercial use.

Sentence Transformer

all-MiniLM-L6-v2

⇧ 62,099,648 Downloads

This model maps sentences and paragraphs to 384-dimensional vectors for semantic search and clustering tasks. It is fine-tuned on 1B sentence pairs using contrastive learning. The model uses JAX/Flax and TPUs for training. It outputs sentence vectors for information retrieval, clustering, and sentence similarity. Text longer than 256 tokens is truncated.

Search Engine

MiniPerplx

⇧ 2,856 Stars

This AI-powered search engine helps you find information across multiple sources. It integrates models like Grok 2.0 for AI-driven answers and APIs for web search, weather, programming, maps, translation, academic papers, product search, and more. It also supports YouTube search, flight tracking, and trending media.

PYTORCH TIP

Optimizing Distributed Training with PyTorch Fault Tolerance and DDP

Fault tolerance is essential in large-scale distributed training, but handling interruptions during training can be challenging. PyTorch provides torchft, a library designed to ensure seamless training without stopping the entire process when an error occurs. This allows you to recover from issues in multi-node or multi-GPU environments without interrupting your work.

Instead of relying on manual error recovery or restarting training jobs, torchft coordinates between different replica groups, automatically resuming training from the point of failure.


Why This Works

torchft uses a lighthouse server and replica managers to maintain synchronization across distributed nodes. This system ensures that training can continue even if there are interruptions in one or more worker nodes. By minimizing the need for manual intervention, it optimizes training efficiency and reduces downtime, making it ideal for long-running, resource-intensive tasks.


from torchft import Manager, DistributedDataParallel
from torchft import Optimizer, ProcessGroupGloo

manager = Manager(pg=ProcessGroupGloo())
model = nn.Linear(2, 3)
model = DistributedDataParallel(manager, model)
optimizer = Optimizer(manager, optim.AdamW(model.parameters()))

for epoch in range(1000):
    optimizer.zero_grad()
    output = model(torch.rand(2, 2))
    loss = output.sum()
    loss.backward()
    optimizer.step()

At Alpha Signal, our mission is to build a sharp, engaged community focused on AI, machine learning, and cutting-edge language models, helping over 200,000 developers stay informed and ahead. We’re passionate about curating the best in AI, from top research and trending technical blogs to expert insights and tailored job opportunities. We keep you connected to the breakthroughs and discussions that matter, so you can stay in the loop without endless searching. We also work closely with partners who value the future of AI, including employers and advertisers who want to reach an audience as passionate about AI as we are.


Our partnerships are based on shared values of ethics, responsibility, and a commitment to building a better world through technology.Privacy is a priority at Alpha Signal. Our Privacy Policy clearly explains how we collect, store, and use your personal and non-personal information. By using our website, you accept these terms, which you can review on our website. This policy applies across all Alpha Signal pages, outlining your rights and how to contact us if you want to adjust the use of your information. We’re based in the United States. By using our site, you agree to be governed by U.S. laws.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Work with us