Skip to main content
Back

New Mistral models outperform larger LLMs in sub-10B category

READ MORE

October 16, 2024 5 min read

Hey,

Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.

We identify and summarize the top 1% news, papers, models, and repos in the AI industry. 

IN TODAY'S SIGNAL

Read time: 5 min 50 sec

🎖️ Top News


📌 Ori

⚡️ Trending Signals


📌 Lambda

📦 Top HF Models

  • Aria: efficient 3.9B parameter model for text, image, and video processing.

  • F5-TTS: non-autoregressive TTS model with multilingual zero-shot capabilities.

  • DepthPro: zero-shot depth prediction with boundary-preserving accuracy and speed.


🧠 Pytorch Tip

  • Identify bottlenecks in Pytorch models using profiler

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

LLM

Mistral AI Introduces Ministral 3B and 8B for Edge Computing

⇧ 1,773 Likes

What's New

Mistral AI has launched two new language models, Ministral 3B and Ministral 8B, aimed at on-device computing and edge use cases. These models set a new frontier in knowledge, commonsense, reasoning, function-calling, and efficiency in the sub-10B category.


Key Highlights

  • Ministral 3B: Achieves scores of 60.9 on MMLU and 72.7 on Winogrande.

  • Ministral 8B: Scores 65.0 on MMLU and 75.3 on Winogrande, demonstrating superior performance in multiple tasks.

  • Context Length: Both models support up to 128k context length.

  • Pricing: Ministral 3B is priced at $0.04/M tokens, while Ministral 8B costs $0.1/M tokens.


Use Cases and Applications

These models cater to local, privacy-first inference needs for applications such as on-device translation, internet-less smart assistants, local analytics, and autonomous robotics. They offer a compute-efficient and low-latency solution suitable for diverse users, from hobbyists to large manufacturing teams. Additionally, they can be integrated with larger models like Mistral Large to efficiently manage multi-step workflows involving function-calling across various contexts.


Performance Metrics

  • For instance, Ministral 3B outperforms the larger Mistral 7B in most benchmarks despite having fewer parameters.

  • Ministral 3B achieves notable accuracy in MMLU and Knowledge & Commonsense tasks, outperforming models like Gemma 2 2B and LLaMA 3.2 3B.

  • Ministral 8B surpasses Mistral 7B in all categories, showcasing its effectiveness in multilingual tasks and GSM8k Maj@8, reflecting its advanced capabilities in reasoning and commonsense understanding.


Availability and Licensing

Both models are available via API with commercial licenses. Model weights for research use are accessible, and Mistral AI offers support for lossless quantization to optimize performance for specific use cases. These features make the models versatile tools for engineers seeking efficient AI solutions in edge computing scenarios.

Read more

Chart your own AI reality with H200 infrastructure

NVIDIA just released the H200 GPU, now available for reservation through Ori Global Cloud.


This GPU boasts 141GB of VRAM, optimized for resource-intensive AI models and high-performance computing.


The H200 accelerates large-scale AI training and inference, offering enhanced memory and processing power for developers.


AlphaSignal readers can now reserve the limited early access.

Secure your access

partner with us

TRENDING SIGNALS

NVIDIA

NVIDIA releases open-source Llama 3.1 70B fine-tune, outperforming GPT-4o and Claude 3.5 on benchmarks

⇧ 8,777 Likes

Local LLM

Lenovo unveils AI Now, an on-device agent,  without cloud reliance powered by Meta's Llama 3.1 for enhanced privacy

⇧ 582 Likes

LLM

Perplexity introduces AI-powered finance tool, offering real-time stock analysis and historical data for developers

⇧ 5,011 Likes

LLM

Unsloth fixes a universal issue in Gradient Accumulation that negatively impacted everyone's training of LLMs

⇧ 772 Likes

LLM

Together AI introduces LoLCATs: Convert existing LLMs to subquadratic variants at LoRA finetune cost

⇧ 266 Likes

New On-Demand 1x, 2x, and 4x NVIDIA H100 SXM GPU Instances

Lambda just added new instances in their public cloud that deliver up to 51% higher performance than PCIe models.


AI developers now have cheaper, flexible access to powerful compute without needing full-node 8x setups.

Get your instance

TOP MODELS

SoTA Multimodal

Aria

⇧10,387 Downloads

Aria helps you build powerful multimodal AI applications. It processes text, images, and videos with a 64K token context window.


The model outperforms competitors on various benchmarks

  • 54.9% on MMMU (multimodal knowledge)

  • 66.1% on MathVista (multimodal math)

  • 92.6% on DocQA (document understanding)

  • 65.3% on LongVideoBench (video comprehension)

  • 73.2% on HumanEval (coding)

Aria uses 3.9B activated parameters per token, making it efficient and fast. It's superior in video and document tasks.

Text-to-Speech

F5-TTS

⇧ NA Downloads

F5-TTS helps you generate natural, expressive speech from text without complex models. It's a non-autoregressive system using flow matching and Diffusion Transformer.


Key features

  • Faster training and inference (RTF 0.15) compared to other diffusion-based TTS models

  • Zero-shot capability and seamless code-switching

  • Speed control efficiency

  • Trained on 100K hours of multilingual data

  • Uses ConvNeXt V2 for text representation

  • Introduces Sway Sampling for improved performance and efficiency

  • Fully open-source with released code and checkpoints

Computer Vision

DepthPro

⇧ NA Downloads

DepthPro helps you generate high-resolution, metric depth maps from single RGB images in 0.3 seconds. It's a zero-shot model that produces sharp, detailed depth estimates without camera metadata.


Key features:

  • 2.25-megapixel output

  • Absolute scale depth prediction

  • Efficient multi-scale vision transformer architecture

  • Trained on real and synthetic data

  • New boundary accuracy metrics for depth maps

  • State-of-the-art focal length estimation

It outperforms previous models in speed and detail preservation, particularly at object boundaries.

PYTORCH TIP

Identify bottlenecks in Pytorch models using profiler

PyTorch Profiler helps you identify performance bottlenecks in your deep learning models. You can track CPU and GPU usage, memory consumption, and operator-level execution times. Use it when optimizing model training or inference pipelines.



import torch
from torch.profiler import profile, record_function, ProfilerActivity

model = torch.nn.Linear(100, 100)
inputs = torch.randn(100, 100)

with profile(activities=[ProfilerActivity.CPU,
                         ProfilerActivity.CUDA],
             record_shapes=True,
             profile_memory=True) as prof:
    with record_function("model_inference"):
        model(inputs)

print(prof.key_averages().table(sort_by="cpu_time_total",
                                row_limit=10))


This code profiles a simple linear model. The profiler records CPU and GPU activities, tensor shapes, and memory usage. The record_function decorator labels specific code sections for easier analysis. This data helps identify potential bottlenecks in even simple operations, allowing developers to optimize their code for better efficiency. CopyRetry


Benefits include faster model execution, reduced memory usage, and improved hardware utilization. It's particularly useful for large-scale models and production environments where efficiency is crucial.

Run code on colab

Thank You.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Advertise