Hey, | Welcome to today's edition of AlphaSignal.
Whether you are a researcher, engineer, developer, or data scientist, our summaries are there to keep you up-to-date with the latest breakthroughs in AI.
Let's get into it!
Lior | |
|
|
IN TODAY'S SIGNAL
|
Top News: Microsoft's New Phi-3 Trending Signals: Karpathy LLM.c, HuggingChat, Gemini 1.5 Top Repos: Mini-Gemini, reflex, LLaMA-Factory Pytorch Tip: Analysis with Profiler
| Read Time: 5 min 03 sec |
|
|
|
Enjoying this newsletter? Please forward it to a friend or colleague. It helps us keep this content free.
|
|
|
|
TOP NEWS
| Language Models | Microsoft Releases Phi-3: An Open LLM That Fits on Your Phone |
⇧ 7129 |
 |
What's New |
Microsoft has launched the Phi-3 series, which includes three sizes: mini (3.8 billion parameters), small (7 billion parameters), and medium (14 billion parameters). These models are designed to run efficiently on both mobile devices and PCs, using advanced datasets to achieve high performance.
Technical Details and Performance:
- Architecture: All models feature a transformer decoder architecture.
- Performance: The mini model achieves 69% on the MMLU and 8.38 on MT-bench, showing performance on par with larger models such as Mixtral 8x7B and GPT-3.5.
- Context Support: Supports a default 4K context length, expandable to 128K through LongRope technology.
Training and Data Strategy: The models are trained on a combination of heavily filtered web data and synthetic data, using a two-phase approach that enhances both general knowledge and specialized skills like logical reasoning.
Architecture and Compatibility: Each model uses the same tokenizer as the Llama-2, ensuring compatibility with existing packages. This design focuses on robustness, safety, and effective interaction in a variety of formats, including chat.
Access and Deployment:
- Availability: The entire Phi-3 series is accessible under an MIT license on the Hugging Face platform, allowing for widespread use and integration.
- Mobile Optimization: The mini model is particularly optimized for mobile use, requiring only about 1.8GB when compressed to 4-bits and efficiently processing over 12 tokens per second on mobile devices like the iPhone 14.
Post-Training and Context Extension: Post-training enhancements have improved the models' abilities in specific domains such as math and coding. The mini model also offers an extended context version capable of handling up to 128K tokens for complex tasks. | Community Feedback |
Following the release of Llama 3 models, this is an unprecedented time for the open-source community.
There is a strong trend of models achieving better performance with smaller sizes. Microsoft anticipates companies to increasingly opt for a an dual approach, using both small and large models depending on the use case. | Community Feedback |
Bindu Reddy: "Phi-3 7B just dropped and beats Llama-3 7B handily. With an MMLU of 75.3, it's coming close to 70B SOTA models!! 🤯 I wouldn't be surprised if we ended up with a 7B model that beats GPT-4 by the end of the year."
Nathan Lambert: "I really hope phi 3 proves us wrong about evaluation doping and it is actually an amazing model. But, being an outlier on log compute <-> MMLU plots is a little sus"
Aaron Ng: "if these scores hold up everyone could have something nearly as good as gpt-3.5-turbo on a phone phi-3-mini q4 should only need something like ~2gb of ram (mixtral 8x7b needs ~22gb for comparison)" |
Read the Technical Report Here |
|
Try phi-3
|
|
|
|
| Get On-demand Instances in Seconds |
Accelerate your training and inference workloads with dedicated GPU instances powered by NVIDIA’s H100 Tensor Core available at Latitude.sh's platform.
With pre-installed AI software, high-performance networking, and enterprise-grade hardware, you get from zero to training your LLMs with just a few clicks. |
|
GET MY GPUs
| partner with us → |
|
|
|
TRENDING SIGNALS
| Development |
| ⇧ 1784
| | Mobile |
| ⇧ 1440 |
| Language Models |
| ⇧ 1391 |
| |
| |
|
|
Understanding More Than Speech: Introducing Audio Events by Speechmatics
| Speechmatics launches Audio Events, a ground-breaking feature that automatically detects and labels non-speech events such as [music], [laughter], and [applause] within audio.
Increase the accessibility of their media content across the globe Reduce the effort of labelling non-speech events within audio Automate all caption generation for more inclusive media
| Request Early Access ↗️ |
|
|
|
TOP OF GITHUB
| Multimodal LLM |
| ☆ 2834 |
Mini-Gemini helps you improve Vision Language Models by combining high and low-resolution visual encoders with large language models from 2B to 34B. You train and fine-tune on specialized datasets to enhance image understanding and generation. Download pre-trained weights and access detailed training scripts from its GitHub repository. |
| Development |
| ☆ 16,309 |
Reflex helps you build full-stack web apps entirely in Python, eliminating the need to learn JavaScript. You can deploy these with a single command or host them on your own server. Reflex supports real-time UI updates and integrates with APIs like OpenAI for machine learning functionalities, streamlining both front and backend development in one language. |
| Finetuning |
| ☆ 18,711 |
LLaMA Factory allows you to fine-tune over 100 large language models efficiently. You can employ methods like supervised fine-tuning, reward modeling, and policy-based optimizations such as PPO, DPO, and ORPO. It supports 16-bit and 32-bit full-tuning and LoRA adjustments, with options for 2/4/8-bit QLoRA to reduce GPU memory use. |
| |
| |
|
|
PYTORCH TIP
| Optimization | Profiler |
The PyTorch profiler is a powerful tool that helps analyze the performance characteristics of your deep learning models. It works by collecting detailed timing information and resource usage statistics while your model is running, allowing you to identify bottlenecks and optimize performance.
You should use the profiler when you need to understand where your model's execution time is being spent, whether on CPU or GPU operations, or when you want to optimize memory usage or identify potential areas for parallelization.
It's particularly useful when working with large, complex models or when you need to deploy your models on resource-constrained devices.
You can use it to:
- Identify slow operations or sections of code that could be optimized or replaced with more efficient alternatives.
- Understand memory usage patterns and potential sources of memory leaks.
- Analyze the impact of different input data or model architecture choices on performance.
|
import torch import torch.nn as nn import torch.profiler
# Define a simple model model = nn.Sequential(nn.Linear(100, 50), nn.ReLU(), nn.Linear(50, 10)).cuda()
# Dummy input input_tensor = torch.randn(32, 100).cuda()
# Set up the profiler with torch.profiler.profile( activities=[torch.profiler.ProfilerActivity.CPU, torch.profiler.ProfilerActivity.CUDA], record_shapes=True, profile_memory=True, with_stack=True) as prof:
# Run the model output = model(input_tensor)
# Print the profiler results print(prof.key_averages().table(sort_by="cuda_time_total"))
| Output:
|
|
|
|
|
Ghita is an editor at AlphaSignal and AI Engineer at Clear Ventures. Experienced in Computer Vision and Language models, she holds 2 masters in Applied Mathematics.
|
|
|
|
Want to promote your company, product, job, or event to 180,000+ AI researchers and engineers?
|
|
Sponsor
|
|
|
|
|