Skip to main content
Back

Mamba-2 is Out: Can it replace Transformers?

MAMBA REPO

June 4, 2024 5 min read

Hey,

Welcome to today's edition of AlphaSignal. 


Whether you are a researcher, engineer, developer, or data scientist, our summaries are there to keep you up-to-date with the latest breakthroughs in AI. 


Let's get into it,


Lior

.

IN TODAY'S SIGNAL

Top News

  • Mamba-2: A new state space model architecture that outperforms Mamba and Transformer++

Snorkel AI

  • New Webinar (free): How to align LLMs to enterprise objectives and policies.

Trending Signals

Top Repos

  • Mora: an open-source version of video generation model Sora.

  • Omost: converting LLMs coding abilities to image generation.

  • gpt-computer-assistant: GPT-4o for MacOS, Windows, Ubuntu.

Python Tip

  • Use the walrus operator (:=) for concise variable assignment within conditional expressions, improving code readability.

Read Time: 5 min 01 sec

Enjoying this newsletter?
Please forward it to a friend or colleague. It helps us keep this content free.

TOP ANNOUNCEMENT

Model Architecture

Mamba-2: A new state space model architecture that outperforms Mamba and Transformer++

⇧ 1981 Likes

What's New

After the success of Mamba-1, which accumulated over 10k stars on GitHub, researchers Tri Dao and Albert Gu introduce Mamba-2.


Mamba is a new state space model architecture showing promising performance on information-dense data such as language modeling, where previous subquadratic models fall short of transformers.


You can use Mamba-2 with PyTorch to build and train neural networks handling information-dense data efficiently.


Core Innovation: Structured State Space Duality (SSD)

Mamba-2's core innovation, SSD, combines SSMs and attention mechanisms. SSD constrains the recurrent matrix 𝐴A to a scalar-times-identity structure, simplifying computations and enhancing hardware efficiency. 


This design enables multi-head SSMs, increasing the state size from 16 in Mamba-1 to 64-256 in Mamba-2, and uses matrix multiplications optimized for GPUs/TPUs.


Performance Improvements

Mamba-2 trains 50% faster than Mamba-1 and handles larger state dimensions. At the 3B scale, Mamba-2, trained on 300B tokens, surpasses Mamba-1 and older transformers in performance. The model significantly outperforms Mamba-1 on tasks like multi-query associative recall (MQAR) due to its larger state sizes.


Architectural Changes

Mamba-2 introduces parallel parameter generation for 𝐴A, 𝐵B, and 𝐶C, enabling tensor parallelism and scaling. It maintains efficient memory usage with a constant state size per channel and leverages matrix multiplications for faster computation. These changes simplify the architecture and improve scalability.


Empirical Results

In language modeling tasks, Mamba-2 shows slightly better scaling compared to Mamba-1, adhering to Chinchilla laws, with faster training times. Pretrained models ranging from 130M to 2.8B parameters are available, trained on datasets like Pile and SlimPajama. Performance is consistent across architectures, with minor variations in zero-shot evaluation results due to evaluation noise.

Key Specifications

  • State Size: Increased from 16 (Mamba-1) to 64-256 (Mamba-2)
  • Training Speed: 50% faster than Mamba-1
  • Scale: Models range from 130M to 2.8B parameters
  • Datasets: Trained on Pile and SlimPajama
  • Evaluation Tasks: Includes MQAR and zero-shot evaluations on various benchmarks

Access

Install Mamba-2 with !pip install mamba-ssm and integrate it with PyTorch. 


Pretrained models can be accessed on Hugging Face for various sequence tasks.


Read the blog post

Mamba repo

New Webinar: How to align LLMs to enterprise objectives and policies 

The biggest challenge enterprise AI teams face is aligning LLMs to domain- and enterprise-specific objectives and policies – and ensuring they’re compliant with organizational standards and ethics.

In this webinar, 

  • we'll introduce an enterprise alignment workflow

  • show how we applied it at top finance, insurance and healthcare companies 

  • and improved the accuracy of their AI assistants and chatbots by 20+ points

Can’t attend live?  All registrants will receive an on-demand recording.

Register now (free)

partner with us

TRENDING SIGNALS

Datasets

HF releases "FineWeb-Edu", a high quality 1.3 trillion token LLM dataset praised by Karpathy and the FineWeb technical report

⇧  4766 Likes

Open Source

You can now instantly replace OpenAI with llama.cpp with just one line of code

⇧ 256 Likes

ChatGPT

OpenAI extends the free ChatGPT with internet access, image analysis, data analysis, file upload and GPTs

⇧ 905 Likes

Video Generation

Deepmind unveils Veo, a video-generation model that creates clips from a single reference image

⇧ 1084 Likes

Industry

NVIDIA CEO unveils next generation AI chips, faster Pandas, and Gen AI interface at Computex 2024 - watch the 15min recap

⇧ 2421 Likes

TOP REPOS

Video Generation

Mora

☆ 1304 Stars

Mora helps you perform generalist video generation tasks using a multi-agent framework. It supports text-to-video, text-conditional image-to-video, video extension, video-to-video editing, and digital world simulation.


Mora is open-source, extendable, and achieves performance close to OpenAI's Sora in various tasks, with videos up to 80 seconds long.

Image Generation

Omost

☆ 4478 Stars

Omost converts LLM coding capabilities into image generation. It employs three pre-trained models based on Llama 3 and Phi 3.


These models are trained using ground-truth annotations, automatic image annotations, Direct Preference Optimization (DPO), and tuning data from OpenAI GPT-4. To deploy, you need an 8GB Nvidia GPU. 

Assistants

GPT Computer Assistant

☆ 1541 Stars

This repo helps you run the ChatGPT MacOS app on Windows and Linux. You can easily install as Python library for this time but use the provided pipeline with native install scripts.


The capabilities of this computer assistant include: 

  • Screen Read
  • Microphone
  • System Audio
  • Memory

PYTHON TIP

Leveraging walrus operator for concise variable assignment and conditional expression

The walrus operator (:=) allows you to assign a value to a variable within a conditional expression. By using this operator, you are ensuring:

  • Improved readability: The code becomes more concise and easier to follow, especially when dealing with complex conditions.
  • Reduced redundancy: You avoid repeating the variable name for assignment and checking.

Here's an example to shows you how you could use this operator:


if number := int(input("Enter a number: "), base=10) > 0:
    print(f"The number is positive.")
else:
    print(f"The number is non-positive.")

In the code above:

  1. number := int(input(...)) combines the input conversion and variable assignment into a single line.
  2. The if statement uses the assigned number within the condition.

This approach can be particularly helpful when you need to use the assigned value immediately within the conditional check.

Ghita is an editor at AlphaSignal and AI Engineer at Clear Ventures. Experienced in Computer Vision and Language models, she holds 2 masters in Applied Mathematics.

Thank You.

Want to promote your company, product, job, or event to 180,000+ AI researchers and engineers?

Sponsor