Skip to main content
Back

Mistral Unveils Small 3: 24B Model 3x Faster Than 70B GPT-4o

TRY NOW

January 31, 2025 6 min read

Hey,

Welcome to AlphaSignal – the most read newsletter by AI developers. 


We bring you the top 1% of news, papers, models, and repos, all summarized to keep you updated on the latest in AI.

IN TODAY'S SIGNAL

Read time: 6 min 03 sec

🎖️ Top News


📌 Lambda


⚡️ Trending Signals



📄 Top Papers

🧠 Python Tip

  • Use joblib.Memory to cache computations and avoid redundant calculations, improving efficiency.

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

Language Model

Mistral AI releases Mistral Small 3, an open source mid-sized model, achieving low latency on local hardware

⇧ 7,639 Likes

What's New

Mistral AI introduces Mistral Small 3, a 24B-parameter open-weight language model designed for low-latency inference. It runs under Apache 2.0, achieves 81% on MMLU, and processes 150 tokens per second. It competes with models three times larger, including Llama 3.3 70B, Qwen 2.5 32B, and GPT-4o-mini, while maintaining a 32K context window for extended input handling.


Key Highlights

  • 24B parameters with a 32K context window.

  • Three times faster than Llama 3.3 70B on the same hardware.

  • Optimized for latency, achieving 150 tokens per second.

  • No synthetic data or RLHF, making it a strong base model for further fine-tuning.

  • Runs efficiently on local hardware, including an RTX 4090 or MacBook with 32GB RAM.


Optimized Inference
Mistral Small 3 reduces the number of layers, cutting down inference time without sacrificing performance. This design enables faster response times in real-time applications, such as function calling, agent workflows, and conversational AI. The model runs locally when quantized, requiring only an RTX 4090 or a MacBook with 32GB RAM.


Performance

Mistral Small 3 matches or outperforms larger open models across key evaluation benchmarks:

  • MMLU (81%): Strong general knowledge and reasoning performance.

  • Instruction following: Competitive with instruction-tuned models three times its size.

  • Code generation and math: Performs at the level of Llama 3.3 70B and GPT-4o-mini.

  • Human evaluations: Assessed using 1,000+ proprietary coding and generalist prompts, showing results comparable to much larger models.


Modify and Fine-Tune with Open Licensing
Mistral AI continues its transition to Apache 2.0, moving away from MRL licensing. You can modify, fine-tune, and deploy the model without restrictions. The model’s pretrained and instruction-tuned checkpoints serve as a foundation for domain-specific customization in legal, medical, and technical applications.

Access and Availability

  • Available on Hugging Face, Ollama, Kaggle, Together AI, and Fireworks AI.

  • Coming soon to NVIDIA NIM, Amazon SageMaker, Groq, Databricks, and Snowflake.


Try now

Unlock Affordable AI Inference with Latest Models

Lambda’s Inference API delivers low-cost AI inference without rate limits. Access models like Llama 3.3, Hermes 3, Qwen 2.5, and LFM-40B. Scale workloads easily with pay-as-you-go pricing.

Why Lambda’s Inference API?

  • No rate limits: Scale effortlessly without bottlenecks.

  • Lowest cost per token: As little as $0.02 per 1M tokens.

  • Top-tier models: Access the latest advancements in AI, all in one place.

  • Flexible pricing: Only pay for what you use.

Start testing, building, and scaling your AI now with simple, flexible pricing. Generate your API key today to get started.

Start now

partner with us →

TRENDING SIGNALS

LLM

AI2 introduces Tülu 3 405B, fully open post-training model optimized for verifiable rewards and scaling, excels in math

⇧ 1,739 Likes

Inference Optimization

NVIDIA unveils DeepSeek-R1 671B model, offers high-speed inference for scalable AI agent experiments on NIM microservice

⇧ 3,194 Likes

LLM Training

Karpathy highlights the emerging frontier of reinforcement learning in LLM training, comparing it to student practice

⇧ 9,264 Likes

AI Industry

Andrew Ng explains how DeepSeek-R1 revolutionizes AI with open-source models and reduced costs and its impact on AI applications

⇧ 13,992 Likes

Generative AI

Google updates Gemini with 2.0 Flash for web and mobile, improving reasoning and image generation capabilities

⇧ 3,425 Likes

TOP PAPERS

Reasoning in LLMs

DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

⇧ 2,846 Likes

Problem

Mathematical reasoning is a challenging task for language models due to its structured nature. Existing open-source models struggle to match the performance of closed models like GPT-4 and Gemini-Ultra. There is a gap in high-quality math datasets and effective training methods to improve mathematical reasoning in open models.

Solution
DeepSeekMath 7B builds on DeepSeek-Coder-Base-v1.5 7B with continued pre-training using 120B math-related tokens from Common Crawl. It introduces Group Relative Policy Optimization (GRPO), a reinforcement learning technique that enhances reasoning while optimizing memory usage. The model also undergoes instruction tuning with chain-of-thought and tool-integrated reasoning.

Results
DeepSeekMath 7B achieves 51.7% on the MATH benchmark and 88.2% on GSM8K with GRPO, outperforming other open models and approaching GPT-4-level performance. The model also improves general reasoning and multilingual math benchmarks, making it a strong open-source alternative for mathematical problem-solving.

Agents

Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments

⇧ 387 Likes

Problem

LLM-powered autonomous agents struggle with real-world tasks due to a lack of high-quality interaction data. Existing models rely on human-annotated datasets, limiting their ability to adapt effectively to diverse environments like coding, web navigation, and desktop tasks.

Solution
Learn-by-Interact is a data-centric framework that enables LLM agents to adapt without human annotations. It synthesizes agent-environment interaction trajectories from documentation and constructs task instructions using a process called backward construction. The framework supports both training-based and in-context learning (ICL) approaches with optimized retrieval methods.

Results
Learn-by-Interact improves agent performance across coding, web, and desktop tasks, achieving up to 12.2% gains in ICL with Claude 3.5 and 19.5% in training with Codestral-22B. Backward construction boosts training performance by 14.0%, and its retrieval pipeline outperforms standard retrieval-augmented generation (RAG). The approach sets a foundation for scalable agent data synthesis.

Model Optimization

Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models

⇧ 1,728 Likes

Problem

Fine-tuning language models on human-generated data is limited by the availability and diversity of high-quality examples. This constraint affects performance, especially in tasks like math reasoning and coding, where correctness can be verified.

Solution
ReSTEM is a self-training method using expectation-maximization. It generates model samples, filters them using binary feedback (correct/incorrect), fine-tunes on the filtered samples, and repeats the process. This iterative approach reduces reliance on human data while improving model performance.

Results
ReSTEM improves performance on MATH reasoning and APPS coding benchmarks, scaling well with model size. It outperforms models fine-tuned only on human data, demonstrating that self-training with feedback can enhance language model capabilities.

PYTHON TIP

Optimizing Machine Learning Workflows with Computation Caching

Repeated computations can slow down data preprocessing and feature engineering in machine learning. Using joblib.Memory, you can cache function outputs and avoid redundant calculations, improving efficiency.


Why This Works

This works by caching the results of expensive function calls to avoid redundant computations. Outputs are stored on disk and reused for identical inputs, saving time and resources. It scales well with large datasets and parameter variations, automatically invalidating outdated caches when code changes. This reduces the overhead of repeated tasks, speeding up iterative workflows and improving overall efficiency in machine learning pipelines.


Example


from joblib import Memory

memory = Memory('cache_dir')

@memory.cache
def expensive_function(x):
    return sum(range(x))

At Alpha Signal, our mission is to build a sharp, engaged community focused on AI, machine learning, and cutting-edge language models, helping over 200,000 developers stay informed and ahead. We’re passionate about curating the best in AI, from top research and trending technical blogs to expert insights and tailored job opportunities. We keep you connected to the breakthroughs and discussions that matter, so you can stay in the loop without endless searching. We also work closely with partners who value the future of AI, including employers and advertisers who want to reach an audience as passionate about AI as we are.


Our partnerships are based on shared values of ethics, responsibility, and a commitment to building a better world through technology.Privacy is a priority at Alpha Signal. Our Privacy Policy clearly explains how we collect, store, and use your personal and non-personal information. By using our website, you accept these terms, which you can review on our website. This policy applies across all Alpha Signal pages, outlining your rights and how to contact us if you want to adjust the use of your information. We’re based in the United States. By using our site, you agree to be governed by U.S. laws.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Work with us