Skip to main content
Back

OpenAI makes 4 major announcements at DevDay

READ MORE

October 2, 2024 6 min read

Hey,

Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.

We identify and summarize the top 1% news, papers, models, and repos in the AI industry. 

IN TODAY'S SIGNAL

Read time: 6 min 25 sec

🎖️ Top News


📌 Ori


⚡️ Trending Signals

📦 Top HF Models

  • Llama-3.2-1B: Compact multilingual  small-scale models.

  • Molmo-7B-D-0924: 7B-parameter vision-language model outperforming larger closed-source alternatives.

  • NVLM-D-72BAI: Multimodal llm outperforms gpt-4v on key vision-language benchmarks.


🧠 Pytorch Tip

  • Implementing Custom Loss Functions and Metrics in PyTorch.

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

OpenAI

OpenAI makes 4 major announcements at DevDay, Realtime API, vision model fine-tuning, Model Distillation and Prompt Caching

⇧ 36,832 Likes

What's New

OpenAI Expands Developer Toolkit with Realtime API, Vision Fine-Tuning, Prompt Caching, and Model Distillation

Key highlights

  • Realtime API enables low-latency speech-to-speech experiences

  • Vision fine-tuning improves GPT-4o's image understanding with minimal data

  • Prompt Caching offers 50% discount on recently seen input tokens

  • Model Distillation allows fine-tuning of cost-efficient models using outputs from advanced models

The Realtime API

  • It allows developers to create fast speech-to-speech interactions using GPT-4o. 

  • It combines speech recognition, text processing, and text-to-speech into a single WebSocket connection. 

  • The API supports six preset voices, handles interruptions automatically, and enables function calling for context retrieval or action triggering during conversations.

Vision fine-tuning

  • Developers can now improve model performance on specific visual tasks using as few as 100 images for finetuning.

  • Grab achieved a 20% improvement in lane count accuracy and 13% in speed limit sign localization using this feature. 

  • Automat increased their RPA agent's success rate from 16.60% to 61.67% for UI element location tasks.

Prompt Caching

  • It will automatically applies a discount (50%) on input tokens the model has recently processed. 

  • It works with prompts longer than 1,024 tokens, increasing in 128-token increments. 

  • Caches typically clear after 5-10 minutes of inactivity and are always removed within an hour of last use.

  • This will also result in faster prompt processing times.

Model Distillation

  • This offers an integrated workflow for fine-tuning smaller, cost-efficient models using outputs from more capable models. 

  • This process includes three main components:

  • Stored Completions: Automatically captures and stores input-output pairs generated by advanced models.

  • Evals (beta): Allows creation and execution of custom evaluations to measure model performance on specific tasks.

  • Fine-tuning: Integrates with Stored Completions and Evals for a streamlined distillation process.

Implementation Details

  • These features integrate into existing workflows through API calls.

  • The Realtime API uses a persistent WebSocket connection, while vision fine-tuning and Model Distillation follow processes similar to text-based fine-tuning.

  • Prompt Caching applies automatically to supported models without requiring changes to API integration.

Read more

Boost AI workflows: $1,000 matched credit from Ori, now available

Ori just released new tools for developers working on AI projects. Fine-tune, train, or run inference on AI models with Ori. 

  • On-Demand GPUs: Access a range of NVIDIA GPUs to accelerate model training and inference. Bring AI models to production faster.

  • Reserved Instances: Get guaranteed availability for consistent workloads and critical applications. Save up to 35% with 1-month minimum commitments.

  • Serverless Kubernetes: Deploy models across multiple regions without cold starts. Pay only for actual usage. Ori handles cluster management and load balancing.

These tools support fine-tuning open source models, training foundational models, and inferencing AI applications.


Alpha Signal readers can now sign up and receive $1000 in credit when spending $1000 on any public cloud service. Terms apply.

Get started

partner with us

TRENDING SIGNALS

Vision Model

Pika unveils version 1.5, enhancing AI-generated video with more realistic motion and physics-defying effects

⇧ 13,826 Likes

Meta

Meta unveils open-source Llama Stack, standardizing AI building blocks across the entire development lifecycle

⇧1,100 Likes

Text to Speech

Gradio releases open-source demo showcasing real-time audio transcription with Whisper Large V3 Turbo

⇧ 480 Likes

AI Code generation

Anaconda launches AI Navigator: Free desktop app runs 200+ AI models locally, ensuring data privacy

⇧ 138 Likes

Video Generation

Luma Labs launches hyperfast video generation, delivering full-quality clips 10x faster in under 20 seconds

⇧ 876 Likes

TOP MODELS

LLM

Llama-3.2-1B

⇧47,429 Downloads

Helps you build multilingual AI assistants and agents. It's available in 1B and 3B sizes with 128k context. The models support 8 languages officially, including English, German, and Hindi.


Key features:

  • Outperforms many open-source and closed chat models on benchmarks

  • Uses GQA for improved inference scalability

  • Trained on up to 9T tokens with a December 2023 knowledge cutoff

  • 1B model achieves 49.3% on MMLU, 3B scores 63.4%

  • Excels in math (77.7% on GSM8K for 3B) and reasoning tasks

It's designed for commercial and research use, focusing on dialogue, retrieval, and summarization.

Multimodal model 

Molmo-7B-D-0924

⇧ 12,961 Downloads

Helps you build vision-language AI systems. It's an open-source model that understands images and text. Based on Qwen2-7B with CLIP as vision backbone, it outperforms GPT-4V on benchmarks.


Key points

  • 7B parameters

  • Trained on 1M curated image-text pairs (PixMo dataset)

  • 77.3 average score on 11 academic benchmarks

  • 1056 Elo rating in human preference evaluation

  • Fully open-source, including future release of training code and data

  • Easy integration with HuggingFace Transformers library

  • Efficient inference with autocast and bfloat16 options

Multimodal model 

NVLM-D-72B

⇧ 193 Downloads

Helps you build powerful multimodal AI applications. It's a family of open-source large language models that handle both text and images. The 72B parameter decoder-only version outperforms GPT-4V on several benchmarks:


Key points

  • MMMU test: 54.9% vs 55.7%

  • MathVista: 65.2% vs 49.9%

  • OCRBench: 852 vs 645

  • Achieves SOTA results in vision-language tasks, rivaling proprietary models. 

You can use it for tasks like visual question answering, image captioning, and multimodal reasoning.

PYTORCH TIP

Implementing Custom Loss Functions and Metrics in PyTorch

You can create losses that aren't available in PyTorch's standard library or design metrics that better reflect your model's performance.


Custom losses and metrics help you tailor model training and evaluation to your task's requirements.


Here's how to implement a custom focal loss:


import torch
import torch.nn as nn

class CustomLoss(nn.Module):
    def __init__(self, weight=0.5):
        super().__init__()
        self.weight = weight

    def forward(self, pred, target):
        return self.weight * torch.mean(
            (pred - target)**2)

def custom_metric(pred, target):
    return torch.mean(torch.abs(pred - target))

# Sample data
pred = torch.randn(5, 3)
target = torch.randn(5, 3)

# Use custom loss
criterion = CustomLoss(weight=0.7)
loss = criterion(pred, target)
print(f"Custom Loss: ")

# Use custom metric
metric_value = custom_metric(pred, target)
print(f"Custom Metric: ")


This custom loss differs from standard MSE in two key ways:


Weighting: It introduces a 'weight' parameter, allowing you to adjust the loss's impact. This helps you control the loss's influence during training.


Flexibility: The forward method can be easily modified to incorporate more complex logic. For example, you could add regularization terms or combine multiple loss components.


Standard PyTorch MSE loss doesn't offer built-in weighting or easy customization. This custom approach lets you fine-tune the loss calculation for your specific task requirements, potentially improving model performance for certain problems.

Run it on colab

Thank You.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Advertise