Skip to main content
Back

NVIDIA dropped new model rivaling GPT-4 and Llama-3.1 405B

READ MORE

October 3, 2024 5 min read

Hey,

Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.

We identify and summarize the top 1% news, papers, models, and repos in the AI industry. 

IN TODAY'S SIGNAL

Read time: 5 min 25 sec

🎖️ Top News


📌 WebAI


⚡️ Trending Signals

⚙️ Top Lectures


🧠 Lecture

  • Andrej Karpathy explains about moving from PyTorch to bare-metal C,  stripping down language model training to its core.

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

LLM

NVIDIA dropped multimodal language model that rivals GPT-4 and Llama-3.1 405B

⇧ 2,457 Likes

What's New

Nvidia has released NVLM 1.0, a family of open-source multimodal large language models. In this family the flagship model is NVLM-D-72B's performance is at par with GPT-4 and Claude 3.5 in across many vision and language tasks specific benchmarks.

Key Highlights

  • Open-source release of model weights

  • On par with Llama 3.1 405B on Math and Coding.

  • Improved text-only performance after multimodal training

  • Hybrid architecture for diverse input processing

  • Under "Creative Commons Attribution Non Commercial 4.0 International" license.

  •  Not allowed for commercial usage.

Architecture and Performance

  • NVLM-D-72B employs a hybrid architecture combining various multimodal processing techniques. 

  • This approach enables effective handling of visual and textual inputs, including meme interpretation, image analysis, and step-by-step math problem-solving. 

  • The model achieves state-of-the-art results on benchmarks like MMMU (58.7/54.9 val/test), MathVista (65.2), and OCRBench (852).

  • NVLM-D 1.0 72B outperforms Llama 3.2 90B on most benchmarks despite having fewer parameters, showing particularly strong advantages in MathVista, OCRBench, and VQA tasks. 

  • Compared to GPT-4V, NVLM-D 1.0 72B demonstrates superior performance across all listed benchmarks, with notable leads in OCRBench, AI2D, and VQA, though it falls short of GPT-4V's MMMU test score.

Text-Only Capabilities

  • A notable feature of NVLM-D-72B is its enhanced text-only performance post-multimodal training. 

  • The model increased accuracy by an average of 4.3 points across text benchmarks, outperforming its original language model backbone. 

  • On text-only tasks, it scores 81.7 on MMLU, 93.2 on GSM8K, and 73.1 on MATH.

Implementation

  • Nvidia provides code for distributed loading across multiple GPUs, allowing efficient use of hardware resources for large-scale deployments.

  • The model supports multi-GPU inference.

Read more

Unlock the real value of AI with local, ownable, and customizable models that you control

webAI just released new features that give developers full control over their AI models. Test solutions immediately, utilize local resources, and deploy securely—all while maintaining data privacy.


webAI enables developers to harness AI power on their own terms.


The platform offers

  • Direct testing and iteration without a dedicated AI team

  • Local compute for fast, secure prototyping at reduced costs

  • Simple deployment to local environments

Learn more

partner with us

TRENDING SIGNALS

Google

Google releases Gemini 1.5 Flash-8B, offering 50% lower price and 2x higher rate limits

⇧ 1,100 Likes

Voice AI

Researchers unveil VoiceRestore: Open-source model restores audio quality, fixing noise and distortions

⇧ 431 Likes

Video Generation

Black Forest Labs unveils FLUX1.1 [pro], achieving 6x faster video generation and topping Artificial Analysis leaderboard

⇧ 1,824 Likes

Open Source LLM

Glaive AI releases updates on the controversial Reflection 70B, showing how the model achieved SoTA benchmark numbers

⇧ 548 Likes

AI File Handling

Notion AI heavily integrates AI introducing file-handling capabilities, enabling developers to extract insights from PDFs and images

⇧ 293 Likes

TOP LECTURES

Web scraping

Crawl4AI: web scraping tool for AI-ready data extraction

⇧ 2,500 Likes

Crawl4AI is a free, open-source web scraping tool for AI applications. You'll learn to extract LLM-friendly data from multiple URLs simultaneously. It offers JSON, cleaned HTML, and markdown outputs. You'll explore features like media tag extraction, link gathering, metadata collection, custom JavaScript execution, and advanced chunking strategies. Crawl4AI helps you streamline data collection for AI projects.

Prompt Engineering

Leaked OpenAI meta prompt: optimizing GPT instructions for better results

⇧ 1000 Likes

This example reveals OpenAI's leaked "meta" prompt for optimizing GPT prompts. You'll learn the structure and key components of effective prompts, including task understanding, reasoning order, example usage, and output formatting. You'll gain insights into preserving user content, handling constants, and creating clear, concise instructions for AI models.

Retrieval Optimization

Tokenization and vector search optimization for RAG systems

⇧ 1,500 Likes

This course teaches you tokenization internals, embedding processes, and RAG optimization. You'll learn how different tokenizers work, tackle common challenges, and measure search quality. The course covers HNSW algorithm tuning and quantization methods. You'll gain skills to improve vector search relevance, speed, and memory efficiency in RAG systems.

TOP LECTURE

RAG

Andrej Karpathy explains about moving from PyTorch to bare-metal C,  stripping down language model training to its core

⇧ 3,100 Likes

Andrej Karpathy recently presented at the GPU MODE workshop, unveiling llm.c - a single C file implementation of large language models. This project strips away PyTorch dependencies, reimplementing core components like arrays, autograd, and distributed training from scratch. llm.c aims for portability, determinism, and efficiency, compiling instantly and running on diverse hardware.


In this talk, you'll learn about

  • Porting PyTorch layers to C

  • Implementing forward and backward passes

  • Memory pre-allocation techniques

  • CUDA and GPU optimizations

  • Mixed precision training

  • Multi-GPU and multi-node setups

  • Performance comparisons with PyTorch

  • Open-source development insights

The project achieved 29% less memory usage and 19% faster training than PyTorch nightly for GPT-2 (1.6B) on an 8XH100 node.

Watch video

Thank You.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Advertise