| Hey, | Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.
We identify and summarize the top 1% news, papers, models, and repos in the AI industry. |
|
|
|
IN TODAY'S SIGNAL
| Read time: 6 min 38 sec | 🎖️ Top News
📌 Ray Summit
⚡️ Trending Signals
📌 Growth |School
🧠 Top Papers
OLMoE: Open-source MoE LLM achieving SOTA performance with efficient parameter usage.
In Defense of RAG in the Era of Long-Context Language Models.
FLUX that Plays Music: Scalable text-to-music generation using rectified flow Transformers.
🧠 Deep Dive
|
|
|
|
If you're enjoying AlphaSignal please forward this email to a colleague.
It helps us keep this content free. |
|
|
|
TOP NEWS
| LLM | Reflection 70B, new open-source LLM surpasses Claude 3.5 Sonnet and GPT-4o on key benchmarks |
⇧ 8,348 Likes |
 |
What's New | Matt Shumer releases a new LLM, Reflection 70B, the world’s top open-source model. It is trained using Reflection-Tuning, a technique to help LLMs to fix their own mistakes. 405B version coming next week.
Performance and Benchmarks
Tops MMLU, MATH, IFEval, and GSM8K benchmarks
89.9% MMLU, 79.7% MATH, 90.1% IFEval
Exceeds GPT-4o across all tested metrics
Achieves over 99% accuracy on GSM8K
Significantly outperforms Llama 3.1 405B Reflection-Tuning Process
The technique separates planning into a distinct step, enhancing chain-of-thought reasoning and producing concise outputs.
This approach addresses hallucination tendencies in current LLMs by implementing a self-correction mechanism before committing to an answer.
Here's how it works 1. During inference, the model generates reasoning within <thinking> tags. 2. If it detects a mistake, it uses <reflection> tags to correct itself before moving forward. 3. Only after this self-correction does the model provide a final answer, enclosed in <output> tags.
Development and Availability Created using Glaive AI's data, Reflection 70B was developed over several weeks.
The 70B model weights are accessible on Hugging Face.
A 405B version, expected to further outperform current leading models, is scheduled for release next week. Technical Considerations
The model has undergone decontamination against multiple benchmarks using LMSys' LLM Decontaminator. Early evaluations show strong performance on tasks like LaTeX error correction and basic counting.
However, the reflection process increases token usage, potentially impacting response speed.
A comprehensive report detailing the process and findings is forthcoming.
The team plans to continue refining the technique, with hints at additional improvements in future iterations.
No, success on an 8B scale yet. | |
|
Read more
|
|
|
|
| Take Your AI/ML Skills to the Next Level at Ray Summit | Get an in-person, intensive day of training at Ray Summit. Enhance your AI toolkit and get everything you need to handle complex AI projects.
Join AI experts and thought leaders in San Francisco on September 30, 2024.
Tickets will sell out, so grab yours today. AlphaSignal readers get 15% off.
|
|
Attend the summit
| partner with us |
|
|
|
TRENDING SIGNALS
| AI Agent |
|
⇧ 10,766 Likes | |
Multi Modal AI |
| ⇧ 533 Likes |
| LLM |
|
⇧ 1,700 Likes | |
AI for Coding |
|
⇧ 2,026 Likes | |
LLM |
|
⇧ 389 Likes | | |
|
|
|
|
|
Use AI to 10x your productivity and efficiency at work
| Struggling to balance work and time? This 3-hour AI & ChatGPT workshop (usually $399) is FREE for the first 100 AlphaSignal readers.
Discover AI strategies to boost productivity, automate tasks, and grow your business 10x faster. |
👉 Save your spot now (seats are filling fast) |
|
|
|
|
TOP PAPERS
| LLM |
| ⇧ 1,590 Likes |
Problem LLMs face trade-offs between performance and cost. Mixture-of-Experts (MoE) models offer a solution, but lack open-source options. OLMOE-1B-7B addresses this gap with 6.9B total parameters, activating only 1.3B per token.
Solution OLMOE-1B-7B, a fully open-source MoE LLM, was trained on 5.1 trillion tokens. It uses 64 small experts per layer, with 8 activated. The model employs dropless token-choice routing and auxiliary losses for stability.
Results OLMOE-1B-7B outperforms models with similar active parameters, matching Llama2-13B's MMLU score (54.1%). After instruction tuning, it surpasses Llama2-13B-Chat and DeepSeekMoE-16B on benchmarks like MMLU, GSM8k, and HumanEval. | |
RAG |
| ⇧ 1,180 Likes
| Problem Recent studies favor long-context LLMs over Retrieval-Augmented Generation (RAG) for question-answering tasks. However, this paper argues that extremely long contexts can diminish focus on relevant information, potentially degrading answer quality.
Solution The authors propose Order-Preserve RAG (OP-RAG), which maintains the original order of retrieved chunks in the context, unlike traditional RAG that orders by relevance.
Results OP-RAG outperforms long-context LLMs and vanilla RAG. Using Llama3.1-70B, OP-RAG achieves 47.25 F1 score on EN.QA with 48K tokens, compared to 34.26 F1 score with 117K tokens for long-context LLM without RAG. On EN.MC, OP-RAG reaches 88.65% accuracy with 24K tokens, surpassing Gemini-1.5-Pro's 85.57% with 188K tokens. |
| Transformers |
| ⇧ 1,500 Likes |
Problem
Existing text-to-music generation models struggle with efficiency and quality. Current approaches use language models or diffusion models to represent quantized waveforms or spectral features, but face computational challenges and limited scalability.
Solution
FluxMusic introduces a rectified flow Transformer architecture for text-to-music generation in latent VAE space. It uses double and single stream attention blocks, multiple pre-trained text encoders, and a modulation mechanism for coarse textual information.
Results
FluxMusic outperforms existing models on automatic metrics and human evaluations. The giant version (2.1B parameters) achieves state-of-the-art results: FAD 1.43, KL 1.25, IS 2.98, and CLAP 0.36 on MusicCaps dataset. Human evaluations show improved overall quality and text relevance compared to AudioLDM 2 and MusicGen. |
| |
| |
|
|
TOP LECTURE
| LLM Inferencing | How the new vLLM achieves 2.7x higher throughput and 5x faster time per output token |
⇧ 349 Likes |
 |
vLLM is the one of the most popular for LLM inferencing library. It is known for being high-throughput and memory-efficient inference and serving engine for LLMs. Today they just released their next version v0.6.0.
vLLM v0.6.0 boosts LLM inference performance significantly
2.7x higher throughput for Llama 8B
5x faster time per output token for Llama 8B
1.8x higher throughput for Llama 70B
2x less time per output token for Llama 70B Key enhancements
Separated API server and inference engine. By eliminating GIL (Global Interpreter Lock) constraints, both components can operate more efficiently without CPU contention, leading to improved performance.
Multi-step batch scheduling
Asynchronous output processing
Object caching and non-blocking data transfers (24% throughput improvement with this) Results Overall, now you get faster, more efficient LLM inference without sacrificing features or usability. |
|
Read more
|
|
|
|
Want to promote your company, product, service, or event to 190,000+ AI researchers and engineers?
|
|
Sponsor
|
|
|
|
|