| Hey, | Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.
We identify and summarize the top 1% news, papers, models, and repos in the AI industry. |
|
|
|
IN TODAY'S SIGNAL
| Read time: 6 min 25 sec | 🎖️ Top News
📌 Ori
⚡️ Trending Signals
📦 Top HF Models
Llama-3.2-1B: Compact multilingual small-scale models.
Molmo-7B-D-0924: 7B-parameter vision-language model outperforming larger closed-source alternatives.
NVLM-D-72BAI: Multimodal llm outperforms gpt-4v on key vision-language benchmarks.
🧠 Pytorch Tip
|
|
|
|
If you're enjoying AlphaSignal please forward this email to a colleague.
It helps us keep this content free. |
|
|
|
TOP NEWS
| OpenAI | OpenAI makes 4 major announcements at DevDay, Realtime API, vision model fine-tuning, Model Distillation and Prompt Caching |
⇧ 36,832 Likes |
 |
What's New | OpenAI Expands Developer Toolkit with Realtime API, Vision Fine-Tuning, Prompt Caching, and Model Distillation
Key highlights Realtime API enables low-latency speech-to-speech experiences
Vision fine-tuning improves GPT-4o's image understanding with minimal data
Prompt Caching offers 50% discount on recently seen input tokens
Model Distillation allows fine-tuning of cost-efficient models using outputs from advanced models
The Realtime API
It allows developers to create fast speech-to-speech interactions using GPT-4o.
It combines speech recognition, text processing, and text-to-speech into a single WebSocket connection.
The API supports six preset voices, handles interruptions automatically, and enables function calling for context retrieval or action triggering during conversations.
Vision fine-tuning
Developers can now improve model performance on specific visual tasks using as few as 100 images for finetuning.
Grab achieved a 20% improvement in lane count accuracy and 13% in speed limit sign localization using this feature.
Automat increased their RPA agent's success rate from 16.60% to 61.67% for UI element location tasks.
Prompt Caching
It will automatically applies a discount (50%) on input tokens the model has recently processed.
It works with prompts longer than 1,024 tokens, increasing in 128-token increments.
Caches typically clear after 5-10 minutes of inactivity and are always removed within an hour of last use.
This will also result in faster prompt processing times.
Model Distillation
This offers an integrated workflow for fine-tuning smaller, cost-efficient models using outputs from more capable models.
This process includes three main components:
Stored Completions: Automatically captures and stores input-output pairs generated by advanced models.
Evals (beta): Allows creation and execution of custom evaluations to measure model performance on specific tasks.
Fine-tuning: Integrates with Stored Completions and Evals for a streamlined distillation process. Implementation Details
These features integrate into existing workflows through API calls.
The Realtime API uses a persistent WebSocket connection, while vision fine-tuning and Model Distillation follow processes similar to text-based fine-tuning.
Prompt Caching applies automatically to supported models without requiring changes to API integration. | |
|
Read more
|
|
|
|
| Boost AI workflows: $1,000 matched credit from Ori, now available | Ori just released new tools for developers working on AI projects. Fine-tune, train, or run inference on AI models with Ori.
On-Demand GPUs: Access a range of NVIDIA GPUs to accelerate model training and inference. Bring AI models to production faster.
Reserved Instances: Get guaranteed availability for consistent workloads and critical applications. Save up to 35% with 1-month minimum commitments.
Serverless Kubernetes: Deploy models across multiple regions without cold starts. Pay only for actual usage. Ori handles cluster management and load balancing.
These tools support fine-tuning open source models, training foundational models, and inferencing AI applications. Alpha Signal readers can now sign up and receive $1000 in credit when spending $1000 on any public cloud service. Terms apply.
|
|
Get started
| partner with us |
|
|
|
TRENDING SIGNALS
| Vision Model |
|
⇧ 13,826 Likes | |
Meta |
|
⇧1,100 Likes | |
Text to Speech |
|
⇧ 480 Likes | |
AI Code generation |
|
⇧ 138 Likes | |
Video Generation |
|
⇧ 876 Likes | | |
|
|
|
|
|
TOP MODELS
| LLM | |
⇧47,429 Downloads | Helps you build multilingual AI assistants and agents. It's available in 1B and 3B sizes with 128k context. The models support 8 languages officially, including English, German, and Hindi.
Key features:
Outperforms many open-source and closed chat models on benchmarks
Uses GQA for improved inference scalability
Trained on up to 9T tokens with a December 2023 knowledge cutoff
1B model achieves 49.3% on MMLU, 3B scores 63.4%
Excels in math (77.7% on GSM8K for 3B) and reasoning tasks It's designed for commercial and research use, focusing on dialogue, retrieval, and summarization. |
| Multimodal model |
|
| ⇧ 12,961 Downloads | Helps you build vision-language AI systems. It's an open-source model that understands images and text. Based on Qwen2-7B with CLIP as vision backbone, it outperforms GPT-4V on benchmarks.
Key points
7B parameters
Trained on 1M curated image-text pairs (PixMo dataset)
77.3 average score on 11 academic benchmarks
1056 Elo rating in human preference evaluation
Fully open-source, including future release of training code and data
Easy integration with HuggingFace Transformers library
Efficient inference with autocast and bfloat16 options | |
Multimodal model |
|
⇧ 193 Downloads | Helps you build powerful multimodal AI applications. It's a family of open-source large language models that handle both text and images. The 72B parameter decoder-only version outperforms GPT-4V on several benchmarks:
Key points
MMMU test: 54.9% vs 55.7%
MathVista: 65.2% vs 49.9%
OCRBench: 852 vs 645
Achieves SOTA results in vision-language tasks, rivaling proprietary models. You can use it for tasks like visual question answering, image captioning, and multimodal reasoning. |
| |
| |
|
|
PYTORCH TIP
| Implementing Custom Loss Functions and Metrics in PyTorch | You can create losses that aren't available in PyTorch's standard library or design metrics that better reflect your model's performance.
Custom losses and metrics help you tailor model training and evaluation to your task's requirements.
Here's how to implement a custom focal loss: |
import torch import torch.nn as nn
class CustomLoss(nn.Module): def __init__(self, weight=0.5): super().__init__() self.weight = weight
def forward(self, pred, target): return self.weight * torch.mean( (pred - target)**2)
def custom_metric(pred, target): return torch.mean(torch.abs(pred - target))
# Sample data pred = torch.randn(5, 3) target = torch.randn(5, 3)
# Use custom loss criterion = CustomLoss(weight=0.7) loss = criterion(pred, target) print(f"Custom Loss: ")
# Use custom metric metric_value = custom_metric(pred, target) print(f"Custom Metric: ")
| This custom loss differs from standard MSE in two key ways:
Weighting: It introduces a 'weight' parameter, allowing you to adjust the loss's impact. This helps you control the loss's influence during training.
Flexibility: The forward method can be easily modified to incorporate more complex logic. For example, you could add regularization terms or combine multiple loss components.
Standard PyTorch MSE loss doesn't offer built-in weighting or easy customization. This custom approach lets you fine-tune the loss calculation for your specific task requirements, potentially improving model performance for certain problems. |
|
Run it on colab
|
|
|
|
Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.
|
|
Advertise
|
|
|
|
|