| Hey, | Welcome to today's edition of AlphaSignal.
Whether you are a researcher, engineer, developer, or data scientist, our summaries ensure you're always up-to-date with the latest breakthroughs in AI.
Let's get into it!
Lior |
|
|
|
IN TODAY'S SIGNAL |
Read Time: 5 min 42 sec |
|
|
|
Enjoying this newsletter? Please forward it to a friend or colleague. It will help us keep this content free. |
|
|
|
TOP PAPER | LLM Optimization | Deepmind Mixture-of-Depths: Speeding Up Models By 50% | ⇧ 2140 ⇆ 386 |  | What's New | Google DeepMind recently unveiled Mixture-of-Depths (MoD), boosting processing speed by up to 50% in tasks like natural language processing and complex sequence prediction.
Most of the compute is wasted because not all tokens are equally hard to predict.
This new method dynamically allocates computation in transformer models, optimizing resource use while ensuring accuracy. It processes complex tokens selectively and skips simpler ones, cutting computational overhead significantly.
Mechanism and Functionality: MoD checks each token's complexity within a sequence, applying computation selectively to those needing deeper processing. This strategy moves away from the traditional approach of uniformly allocating computation across all tokens.
Skipping certain layers for specific tokens, MoD lowers the total floating-point operations per second (FLOPs) needed, making computation more efficient.
Performance Metrics: - Compute Savings: MoD cuts FLOPs by 50% during post-training sampling, marking a significant boost in efficiency.
- Training Performance: It keeps accuracy on par with baseline models while using fewer resources, proving its efficiency.
- Speed Improvement: MoD speeds up processing by up to 50% in certain tasks, improving model responsiveness.
Integration and Application: - Compatibility: MoD fits smoothly with existing transformer setups, including those with Mixture of Experts (MoE) technology for even better efficiency.
- Hardware Optimization: It uses a static computation graph, ensuring predictable compute loads and better hardware use.
Accessibility: DeepMind provides detailed guides and source code. | Why This Matters | This paper is another reminder that LLMs are still in their early days: slow, expansive and inefficient. Creating cheap and fast models will open up a world of possibilities such as the ability to run models locally on our phones and GPUs. It could also drastically decrease the cost behind training and running LLMs. | Community Feedback | Teknium: "Does this still limit the maximum amount of compute useable though?" Sandya Mannarswamy: "This reminds me of speculative decoding strategies, Instead of two different models, here it is intra-model, choose a simpler expert (identity fn, or even others but cheaper than full fledged expert).."
Henk Poley: "I wonder if during training you could (ab)use this, and only actually train (or with a higher weight) on tokens mispredicted by the first X layers.
Potentially like, only train the first layers, or more strongly train latter layers." | | Read the paper |
|
|
|
 | Assembly AI New Multilingual Speech Model: Transcribes 1 hour of Audio in 38 sec. | Introducing Universal-1: AssemblyAI’s most capable and highly trained speech recognition model yet.
Trained on over 12.5 million hours of multilingual audio data, Universal-1 achieves best-in-class speech-to-text accuracy, reduces word error rate and hallucinations and improves timestamp estimation as we continue to raise the bar as the industry-leading Speech AI provider.
Universal-1 is 14% more accurate than models like Whisper and >22% more accurate than APIs from Azure/AWS/Google. It achieves up to 30% fewer hallucinations than seq2seq models and can transcribe an hour long audio in only 38 seconds.
Universal-1 is available today for English and Spanish, with French and German to follow soon. Sign up for AssemblyAI’s API to access Universal-1 for $0.37/hour of audio, with volume discounts available. | | Learn more | partner with us |
|
|
|
TOP ANNOUNCEMENTS | GPT Developers | | OpenAI is rolling out new dashboards, metrics, and integrations in its fine-tuning API to give developers better control, alongside introducing new options for building custom models. This update focuses on providing tools for more precise model optimization, aiming to cut costs, reduce latency, and enhance accuracy. | ⇧ 1504 ⇆ 304 | | RAG | | Cohere introduces Command R+, a RAG-optimized LLM for enterprise use, supporting 10 languages and excelling in retrieval, tool use, and complex workflow automation. It's first available on Azure, with a 128k-token context window, priced at $3.00/$15.00 per M input/output tokens. | ⇧ 432 ⇆ 128 | | Finance | | In a presentation earlier this month, the venture-capital firm Sequoia estimated that the AI industry spent $50 billion on the Nvidia chips used to train advanced AI models last year, but brought in only $3 billion in revenue. | | |
| |
|
|
Imagine an AI... that can type anywhere you can on macOS with full context on what's on your screen | Omnipilot brings AI to every Mac app, using the app's context to provide intelligent assistance. Invoke it with a shortcut to supercharge writing, email, and getting answers.
Try it now ↗️ |
|
|
|
TOP PAPER | Language Models | Apple's New ReALM Models Surpass GPT-4 in Conversational AI | ⇧ 1788 ⇆ 350 |  | What's New | Apple Research has unveiled language models from 80M to 3B parameters that excel in conversational understanding and on-screen task interpretation. These models, known as ReALM, demonstrate significant advancements in AI conversational interfaces.
Objective and Training: The development focuses on enhancing AI assistants' ability to process complex dialogues and understand on-screen and background activities. By incorporating diverse datasets, including conversational, synthetic, and on-screen data, the models aim for a comprehensive understanding of user interactions.
Performance Highlights: - ReALM models perform competitively, often outshining GPT-4 in handling dialogues and interpreting on-screen content.
- Notably, the ReALM-250M model achieves impressive results:
- Conversational Understanding: 97.8
- Synthetic Task Comprehension: 99.8
- On-Screen Task Performance: 90.6
- Unseen Domain Handling: 97.2
These metrics underscore the model's proficiency in domain-specific queries and its robust performance across various tasks, including those involving visual elements on the screen.
Innovative Methods: The introduction of a novel encoding technique stands out, converting diverse entity types into a comprehensible text format for the AI. This approach enables the models to efficiently process and understand complex user requests without relying on advanced image recognition, streamlining interactions in multifaceted scenarios. | Why This Matters | It's a reminder that Apple, the sleeping giant, can quickly catch up and stay relevant in the AI industry. Many forget that apple was one of the first companies to deploy AI models at scale. | Community Feedback | Nicholas Brennan: "I’d say there’s a roughly 0% chance Apple releases a consumer product in the near term that beats GPT-4 in any meaningful manner" Axel Darmouni: "Perhaps due to red teaming. ReaLM paper does not mention any sort of safety training, and despite the controversy Google’s Gemini was taking a lot of precaution If they want to use a LLM on device, they need it to be as safe as possible from any jailbreaking attempts"
Sebastian Raschka: "I'd say Q&A accuracy is the most convenient benchmark. But the real tests are conversational benchmarks.."
| | Read the paper |
|
|
|
Ghita is an editor at AlphaSignal and AI Engineer at Clear Ventures. Experienced in Computer Vision and Language models, she holds 2 masters in Applied Mathematics. |
|
|
|
Want to promote your company, product, job, or event to 175,000+ AI researchers and engineers? | | Sponsor |
|
|
|
|