Skip to main content
Back

Meet Llama 3.2: Meta's New Lightweight Model That Can See

READ MORE

September 26, 2024 6 min read

Hey,

Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.

We identify and summarize the top 1% news, papers, models, and repos in the AI industry. 

IN TODAY'S SIGNAL

Read time: 6 min 2 sec

🎖️ Top News


📌 Galileo

⚡️ Trending Signals

📌 Speechmatics


📦 Top HF Models


🧠 Pytorch Tip

  • Custom data structures using Python's collections.abc for efficient handling of high-dimensional tensors

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

LLM

Meta AI Releases Opensource Multimodal AI Models Llama 3.2

⇧ 6,682 Likes

What's New

Meta has expanded its open-source Llama model family with Llama 3.2, introducing multimodal capabilities and lightweight versions for edge devices. 


This release includes vision-enabled models (11B and 90B parameters) and compact text-only models (1B and 3B parameters) designed for mobile and edge computing.


Key highlights

  • Multimodal models support image reasoning tasks

  • Lightweight models optimized for on-device use with 128K token context

  • Llama Stack API introduced for simplified AI application development

  • Models available for immediate use on various cloud and on-device platforms

  • The multimodal models integrate vision capabilities through adapter weights, allowing image processing without altering text performance.

  • This architecture enables them to function as drop-in replacements for previous text-only versions. 

Performance

  • 11B vision model outperforms Claude 3 Haiku across multimodal benchmarks

  • 90B model competes with GPT-4o mini

  • 3B text model surpasses Gemma 2 2.6B and Phi 3.5-mini in tasks like summarization and instruction following

Training and Architecture

  • Lightweight models were created using pruning and knowledge distillation techniques, with larger Llama models serving as teachers. 

  • hese models support a 128K token context window, enhancing their capabilities for longer text processing.

  • Llama Stack provides a standardized interface for customizing Llama models and deploying them across various environments. 

The models are optimized for mobile hardware through partnerships with key chip manufacturers. Meta is actively exploring quantized variants for improved efficiency, though current releases use BFloat16 numerics.


Model Access

  • Developers can access these models through multiple channels:

  • Direct download from llama.com and Hugging Face

  • Cloud platforms like AWS, Google Cloud, and Microsoft Azure

  • On-device deployment via PyTorch ExecuTorch

  • Single-node distribution through Ollama

  • It offers APIs for inference, tool use, and retrieval-augmented generation (RAG), simplifying the development of AI applications. 

Read more

Unlock the Secrets of RAG Systems with Galileo's Free eBook: Mastering RAG

Galileo just released a new FREE eBook: Mastering RAG - A Developer's Guide to Enterprise-Grade RAG Systems


Download now to access 200 pages of technical content covering

  • Chunking strategies

  • Embedding and reranking model selection

  • Vector database comparisons

  • RAG architecture best practices

  • Testing and evaluation methods

Key topics include

  • Mitigating hallucinations

  • Optimizing chunking techniques

  • Choosing the right vector database

  • Overcoming common RAG implementation challenges

  • Preparing your system for production

  • Boosting performance

Get yours today

partner with us

TRENDING SIGNALS

Multimodal AI models

Allenai introduces open-source multimodal AI models Molmo outperforming proprietary systems using 1000x less data

⇧ 2,100 Likes

LLM Inference

Groq started supporting the newly released Llama 3.2 in GroqCloud

⇧750 Likes

Google

Google unveils AlphaChip, an AI-driven chip design method transforming TPU and CPU development

⇧ 902 Likes

LLM

LlamaIndex releases guide for advanced RAG on Excel files using LLMs, outperforming GPT-4 on table recognition

⇧ 1,400 Likes

OpenAI

ChatGPT's Advanced Voice feature goes viral, tuning guitar with potential implications for audio-based AI tasks

⇧ 9,200 Likes

Power your Conversational AI with this Real-Time Speech API

Speechmatics just released an updated speech-to-text API for Conversational AI.

It offers:

  • High-precision transcription

  • Minimal latency

  • Support for multiple languages globally

The API enables responsive voice interactions in real-time.

Try it today [Free]👉

TOP MODELS

OCR

GOT-OCR2_0

⇧133,528 Downloads

Helps you extract text from PDFs and images with high accuracy and flexibility.


You can

  • Get plain text, formatted output, or fine-grained OCR with bounding boxes and color info

  • Process multiple cropped regions in a single image

  • Handle various document types and complex layouts

  • Built on Hugging Face Transformers, it runs efficiently on NVIDIA GPUs. The model improves on its predecessor, addressing key issues and enhancing capabilities.

LLM

Llama-3_1-Nemotron-51B

⇧ 496 Downloads

Helps you build efficient, high-quality language applications. It's a 51B parameter model derived from Llama-3.1-70B, optimized for single H100-80GB GPU use.

Key features

  • Neural Architecture Search for memory efficiency

  • 80.2% on MMLU (5-shot), 91.43% on GSM8K (5-shot)

  • 8192 token context length

  • Supports English, coding, and other languages

  • Licensed for commercial use

  • Trained on 40B tokens, data cutoff 2023

  • Available through Hugging Face Transformers

LLM

Mistral-Small-Instruct-2409

⇧ 4,753 Downloads

Helps you build powerful AI applications with less computational resources.


This 22-billion parameter model supports:

  • Multilingual tasks

  • Tool use and function calling

  • 128K sequence length

  • 32,768 token vocabulary

It's designed for instruction-following and handles various NLP tasks efficiently. You can access it freely on Hugging Face Hub for non-commercial use. 

The model balances performance and scalability, positioning between Mistral NeMo 12B and Mistral Large 123B. Its compact size makes it suitable for resource-constrained environments while maintaining high-quality output.

PYTHON TIP

Custom data structures using Python's collections.abc for efficient handling of high-dimensional tensors

Python's collections.abc module lets you create tailored classes that behave like built-in containers. This approach optimizes memory usage and computation speed for specific tensor operations.


This class supports indexing and custom methods like sum_along_axis. It's useful for tasks involving 3D data, such as video processing or 3D image analysis. The benefits include intuitive access patterns and specialized operations while maintaining compatibility with Python's sequence protocols.


Here's a simple example of a custom 3D tensor class:


from collections.abc import Sequence
import numpy as np

class Tensor3D(Sequence):
    def __init__(self, data):
        self._data = np.array(data)
        
    def __getitem__(self, index):
        return self._data[index]
    
    def __len__(self):
        return len(self._data)
    
    def sum_along_axis(self, axis):
        return np.sum(self._data, axis=axis)

# Usage
t = Tensor3D([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
print(t[0])  # Access first 2D slice
print(t.sum_along_axis(0))  # Sum along first axis



In above, the collections.abc module helps by providing the Sequence abstract base class.


This

  • Enforces a consistent interface (getitem and len)

  • Adds extra functionality automatically (contains, iter, reversed)

  • Allows custom methods (sum_along_axis) alongside standard operations

  • Improves interoperability with other Python code

This creates a tailored 3D tensor structure that's efficient and compatible with Python's sequence protocols.

Get the google colab

Thank You.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Advertise