| Hey, | Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.
We identify and summarize the top 1% news, papers, models, and repos in the AI industry. |
|
|
|
IN TODAY'S SIGNAL
| Read time: 5 min 16 sec | 🎖️ Top News
📌 Galileo
⚡️ Trending Signals
🛠️ Top of Github
mlc-llml: Universal LLM Deployment Engine with ML Compilation.
surya: OCR, layout analysis, reading order, table recognition in 90+ languages.
TEN-Agent: world’s first real-time multimodal agent integrated with the OpenAI Realtime API, RTC, and features weather checks, web search, vision, and RAG capabilities.
🧠 Lecture
|
|
|
|
If you're enjoying AlphaSignal please forward this email to a colleague.
It helps us keep this content free. |
|
|
|
TOP NEWS
| Hybrid-SSM model | Zyphra Releases Zamba2-7B: A Hybrid SSM-Attention Model Outperforming Current 7B Leaders |
⇧ 965 Likes |
 |
What's New |
Zyphra has unveiled Zamba2-7B, a 7 billion parameter language model that combines Mamba2 blocks with shared attention layers. This open-source model surpasses other leading models in the ≤8B parameter range, including Mistral-7B, Gemma-7B, and Llama3-8B, in both quality and inference speed.
Key highlights
Outperforms Mistral-7B, Gemma-7B, and Llama3-8B in benchmarks
25% faster time to first token, 20% higher tokens per second than Llama3-8B
Trained on 3 trillion tokens with 100B token "annealing" phase
Open-source under Apache 2.0 license
Instruct-tuned version available
Architecture The Zamba2-7B architecture features a backbone of Mamba2 layers interspersed with two shared attention layers in an ABAB pattern.
This design allocates more parameters to the Mamba2 backbone while preserving cross-sequence dependencies.
LoRA projectors applied to shared MLPs allow for depth-specific specialization.
Zamba2-7B Inference Performance
SOTA inference efficiency, including latency, throughput and memory usage.
Roughly 4 times the throughput of an equal-parameter transformer block.
One of the key reason for such performance is that Mamba blocks only have small hidden states to store and don't require a KV-cache. The model only stores KV states for shared attention block invocations, reducing memory usage.
Training Details
The model's training involved a 3 trillion token dataset, followed by a 100 billion token "annealing" phase using high-quality data. This approach contributes to its strong performance on a per-training-token basis and across standard benchmarks for the ≤8B parameter class.
The training process utilized 128 H100 GPUs over approximately 50 days, employing Zyphra's internal framework built on Megatron-LM. This demonstrates the feasibility of pushing the 7B scale frontier with moderate resources.
Access and availability Developers can access Zamba2-7B through Huggingface and also from NVIDIA’s NIM site. The model is designed for on-device use and consumer GPUs, making it accessible for various applications. | |
|
Read more
|
|
|
|
| Learn from Nvidia, HP, Databricks, and more | Nvidia, HP, Databricks, ServiceNow, and Twilio just announced the premier conference for GenAI:
Designing GenAI stacks for enterprise scale
AI governance, evaluation, and observability strategies
Production-ready GenAI app best practices Developers can now access these key learnings to enhance their AI implementations and workflows.
|
|
Free registration
| partner with us |
|
|
|
TRENDING SIGNALS
| Video Generation |
|
⇧ 1,727 Likes | |
Multimodal Model |
|
⇧ 1,400 Likes | |
LLM |
|
⇧ 560 Likes | |
Perplexity |
|
⇧ 1,000 Likes | |
Google |
| ⇧ 1,200 Likes
| | |
| |
|
|
TRENDING REPOS
| LLM Deployment | |
☆ 18,910 Stars |
This repo helps you deploy LLMs across diverse hardware: GPUs (NVIDIA, AMD, Apple, Intel), CPUs, and web browsers. It uses ML compilation to optimize performance. You can run models like Llama 2 and Vicuna on your chosen device with OpenAI-compatible APIs. |
| OCR |
|
| ☆ 11,200 Stars |
helps you do OCR, text detection, layout analysis, reading order detection, and table recognition for documents. It works in 90+ languages with high accuracy. You can process PDFs and images. It outperforms Tesseract in text detection (82.1% vs 63.1% precision) and matches Google Cloud Vision in OCR accuracy (97% similarity). The models use modified EfficientViT and Donut architectures, trained on diverse datasets. You can install it with pip and use it via CLI, Python API, or a Streamlit GUI. |
| Code Manager |
|
☆ 711 Stars |
helps you build high-performance multimodal AI applications with real-time interactions. You can integrate OpenAI's Realtime API, RTC, and custom extensions in C++, Go, or Python. It supports edge-cloud deployment, multi-language processing, and real-time agent state management. The drag-and-drop interface simplifies complex AI system creation, combining audio-visual tools, databases, and RAG. |
| |
| |
|
|
| LECTURE |
AI Evolution |
Andrej Karpathy on AI evolution: from self-driving cars to AGI |
⇧ 5,600 Likes |
 |
Andrej Karpathy, former Tesla Autopilot leader and OpenAI founding member, shares insights on AI's evolution in a No Priors podcast interview. He discusses the shift from traditional programming to "Software 2.0", where neural networks and data define software behavior.
He talks about on the importance of extremely smaller-sized distilled models where even 1Bn param model should be useful enough.
Karpathy explores vision-based approaches in self-driving cars, drawing parallels between autonomous vehicles and potential AGI development. He emphasizes data's crucial role in modern software and AI's impact on various sectors.
In this discussion, you'll learn about Parallels between human cognition and AI models
Building high performance small models
Software 2.0 concept and its implications
Vision-based AI systems in autonomous driving
Comparison of Tesla and Waymo approaches
AGI development parallels with self-driving tech
AI's economic impact, particularly on transaction costs
Data's role as the new 'code' in AI-driven software
AI's potential in reshaping education and productivity |
|
Watch it
|
|
|
|
Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.
|
|
Advertise
|
|
|
|
|