Hey, | Welcome to today's edition of AlphaSignal.
Whether you are a researcher, engineer, developer, or data scientist, our summaries are there to keep you up-to-date with the latest breakthroughs in AI.
Let's get into it!
Lior |
|
|
|
IN TODAY'S SIGNAL
| Top Paper:
Top Signals: Top of HF:
- MMLU-Pro, a robust multi-task understanding dataset for large language models.
- MistoLine, an SDXL-ControlNet model for high-quality line art images.
- FineWeb, optimized CommonCrawl dataset for superior LLM performance.
Tutorial: |
Read Time: 5 min 32 sec |
|
|
|
Enjoying this newsletter? Please forward it to a friend or colleague. It helps us keep this content free.
|
|
|
|
TOP PAPER
| Language Models | Models Are Converging Towards the Same Representation of the World |
⇧ 1871 Likes |
 |
What's New | Are AI models converging in the way they represent the world? This recent paper says yes.
The authors analyze several language and vision models, their learned latent spaces and the way they measure distances between data points, and conclude that there is a noticeable and growing alignment between all of them.
The paper makes the hypothesis that different models are all trying to arrive at the same representation of reality. This shared statistical representation of the world is named after Plato's concept of an ideal reality, and referred to as the platonic representation.
What is intriguing is that neural networks, even if they are trained with different objectives on different data and modalities, seem to be converging to a shared statistical model of reality. Given the variability of training settings (loss functions, datasets, modalities, architectures...), it is not obvious why this would be the case.
You might wonder how we can measure the similarity between two latent spaces, or representations, for two different foundational models.
This paper focuses on vector embeddings and looks at the similarity structure they induce, meaning how the distance/similarity between data points is measured for each model.
In other words, this paper looks at how each foundational model measures similarity between its data points, or its kernel. It then introduces a kernel-alignment metric that determines how close two kernels of two different foundational models are.
Some results that argue in favor of the hypothesis include:
Several selective pressures are identified as driving this convergence, including the scaling of model size, data diversity, and the need to solve a wide range of tasks. | |
Why it Matters |
There are broader implications of this potential convergence than just scientific curiosity. This work shows the potential for more efficient transfer learning, better integration of multimodal data, and the development of more general AI systems. | Access
| |
Community Feedback |
Anıl If this is true, AI models will be able to seamlessly integrate and process different types of data (e.g., text, images, audio) within a single model. |
Pawel Pachniewski They're converging on the actual reality that consists of all the regularities we know through physics. This is to be expected. It's not just chaos and noise out there. And yes, different objectives/modalities etc. should matter, if these models don't get stuck in local minima and basically get far enough to get invariants of reality. |
|
Read the paper
|
|
|
|
| Bank at the speed of AI. | Struggling to get a clear picture of your company’s finances? Constantly chasing spend? Losing hours to tedious banking* tasks?
When you’re building in the fastest-moving industry in the world, you need a bank account that doesn’t hallucinate. Mercury simplifies all your financial workflows by powering them from the bank account.
Mercury enables you to tackle banking tasks in a click, ensure spend lines up with company goals, and process bills faster with built-in AI. Plus, you’ll unlock exclusive perks for AWS, Google Cloud, GoCharlie, and more.
When you sign up and deposit $10K in your first 90 days, you’ll get a $500 cash bonus.
Apply online in 10 minutes. |
|
Apply now
| partner with us → |
|
|
|
TRENDING SIGNALS
| ChatGPT |
|
⇧ 8193 Likes | |
Open Source |
| ⇧ 1131 Likes |
| Model Interpretation |
| ⇧ 1093 Likes
| | Chatbots |
| ⇧ 4192 Likes |
| Open Source |
| ⇧ 819 Likes |
| |
| |
|
|
Discover the Power of GenAI for Intelligent Document Processing
| Join this webinar to discover the superpowers of AI and GenAI for document processing:
Building your own AI models with as little training as possible
Processing complex emails with UiPath Communications Mining and CommPath LLM
Out-of-the-box document processing with UiPath Document Understanding and DocPath LLM in action |
Save your seat ↗️ |
|
|
|
TOP HF MODELS/DATASETS
| LLM Benchmarking | |
A more robust and challenging massive multi-task understanding dataset tailored to more rigorously benchmark large language models' capabilities. This dataset contains 12K complex questions across various disciplines. | | Image Generation |
|
An SDXL-ControlNet model that can adapt to any type of line art input, demonstrating high accuracy and excellent stability. It can generate high-quality images (with a short side greater than 1024px) based on user-provided line art of various types, including hand-drawn sketches, different ControlNet line preprocessors, and model-generated outlines. |
| Web Dataset |
|
The FineWeb dataset consists of more than 15T tokens of cleaned and deduplicated english web data from CommonCrawl. The data processing pipeline is optimized for LLM performance and ran on the datatrove library.
Models trained on this dataset outperform models trained on other commonly used high quality web datasets (like C4, Dolma-v1.6, The Pile, SlimPajama, RedPajam2). |
| |
| |
|
|
TUTORIAL
| Fine-tuning | Learn how to fine-tune PaliGemma: Google's latest open multimodal model |
| PaliGemma, released by Google in May 2024, is a Large Multimodal Model (LMM). You can use PaliGemma for Visual Question Answering (VQA), to detect objects on images, or even generate segmentation masks.
While PaliGemma has zero-shot capabilities – meaning the model can identify objects without fine-tuning – such abilities are limited. Google strongly recommends fine-tuning the model for optimal performance in specific domains.
The training data consists of 90 pairs of images and long captions describing them. The only parts that is fine-tuned here are the attention layers of the language model, while the other parameters are kept frozen.
This notebook uses the model reference implementation from big_vision: - Install dependencies, download model checkpoint and training data.
- Load the model onto GPU devices.
- Prepare the input to the model for training and inference.
- Fine-tune the model and inspect output in validation split.
|
|
Fine-tune paligemma
|
|
|
|
Ghita is an editor at AlphaSignal and AI Engineer at Clear Ventures. Experienced in Computer Vision and Language models, she holds 2 masters in Applied Mathematics.
|
|
|
|
Want to promote your company, product, job, or event to 180,000+ AI researchers and engineers?
|
|
Sponsor
|
|
|
|
|