Skip to main content
Back

Liquid Foundation Models: Brings A New Architecture

READ MORE

October 1, 2024 5 min read

Hey,

Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.

We identify and summarize the top 1% news, papers, models, and repos in the AI industry. 

IN TODAY'S SIGNAL

Read time: 5 min 15 sec

🎖️ Top News


📌 Pythagora

⚡️ Trending Signals

🛠️ Top of Github

  • ultralytics: Helps you implement Ultralytics YOLO11, the cutting-edge, SOTA model for object detection and tracking, instance segmentation.

  • TTS: a deep learning toolkit for Text-to-Speech

  • aider:  AI pair programming in your terminal


🧠 Recommended Lecture

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

LLM

Liquid AI Unveils New Architecture for Language Models, outperforming Llama 3.2 in similar size classes

⇧ 3,214 Likes

What's New

Liquid AI has introduced Liquid Foundation Models (LFMs), a new series of generative AI models that challenge traditional transformer-based architectures. These models aim to achieve state-of-the-art performance while reducing memory footprint and improving inference efficiency.


Key Highlights:

  • Three models released: 1.3B dense, 3.1B dense, and 40.3B MoE (12B activated)

  • LFM-1B and LFM-3B outperform larger transformer models in benchmarks

  • 32k token context length, effective across the entire range

  • Reduced memory footprint, especially for long inputs

  • The models aren’t opensourced. 

Performance

  • The 1B and 3B models deliver impressive performance, outperforming Llama 3.2, Gemma 2, and Phi 1.5 models in their respective size classes across benchmarks like MMLU-Pro and Hellaswag.

  • The LFM-1B model achieves a 58.55 MMLU score (5-shot), while the LFM-3B scores 66.16. The LFM-40B competes with larger models, scoring 78.76 on MMLU.

Architecture


LFMs use structured operators and adaptive linear units, allowing customization for different modalities and hardware requirements. 


Memory Efficiency and Context Length

  • LFMs demonstrate superior memory efficiency, particularly for long inputs. Unlike transformer models where KV cache grows linearly with sequence length, LFMs maintain a minimal memory footprint.

  • The 32k token context length, confirmed effective by the RULER benchmark, enables long-context tasks on edge devices, opening new possibilities for document analysis and context-aware applications.


Application and Access

  • Liquid AI plans to expand LFMs to very long contexts and introduce new capabilities.

  • You can try them today on Liquid Playground, Lambda (Chat UI and API), Perplexity Labs, and soon on Cerebras Inference.

Read more

First AI dev tool that builds production-ready apps

Pythagora has just released a groundbreaking tool for building full-stack, production-ready apps through conversation. It goes beyond typical AI demo app creators, allowing developers to focus on ideas and creative decisions while AI handles the codebase.


Data scientists, DevOps professionals, and engineers unfamiliar with web app development can now describe their desired application in natural language. Pythagora manages the development process, requesting feedback when needed.


The tool aims to streamline app creation for those with limited web development experience. It offers a new approach to AI-assisted development, emphasizing practical, deployable results.


Pythagora v1 is now available for early access. Developers can sign up to build their first app at no cost.

Start building now

partner with us

TRENDING SIGNALS

OpenAI

OpenAI announces 98% cost reduction for GPT-4 to 4o mini, boosting token processing 50x, introduces prompt caching

⇧ 6,800 Likes

Microsoft

Microsoft launches redesigned Copilot with Voice, Vision, and Chain of Thought capabilities

⇧1,523 Likes

Google

Google rolls out simplified AI Studio mobile experience, bringing models to forefront

⇧ 1,400 Likes

AI Code generation

New GitHub repo "o1-engineer" goes viral, a CLI tool for streamlining workflows with AI-powered code generation and editing

⇧ 2,200 Likes

AI Audio

ElevenLabs upgrades ElevenReader app, enabling AI-powered listening for its expansive text library

⇧ 380 Likes

TRENDING REPOS

Object detection

ultralytics

☆ 29,210 Stars

Helps you implement YOLO11 for object detection, segmentation, classification, and pose estimation. It offers pre-trained models with state-of-the-art performance (e.g., 54.7 mAP for YOLO11x on COCO). You can train, validate, and deploy models using Python or CLI. It supports various tasks and integrates with platforms like Roboflow and ClearML.

 Text-to-Speech

TTS

☆ 34,102 Stars

Helps you build and deploy text-to-speech systems. It provides pre-trained models for 1100+ languages, tools for training custom models, and a framework for research. You can clone voices, perform voice conversion, and generate speech with sub-200ms latency. It supports 16 languages out-of-the-box and includes implementations of Tacotron2, Glow-TTS, FastPitch, and various vocoders.

Code Generator

aider

☆ 19,700 Stars

Helps you pair program with LLMs directly in your terminal. It edits code in your local git repo, works with multiple files, and supports GPT-4 and Claude 3.5 Sonnet. Aider auto-commits changes, handles various programming languages, and uses a repo map for larger codebases. It scored highly on the SWE Bench, solving real GitHub issues from popular open-source projects.

TOP LECTURE

AI Agent

Google Research Engineering Team Lead dicsusses a future of very intelligent AI agents

⇧ 625 Likes

Frederic Besse, Research Engineering Team Lead at Google DeepMind discusses "agents", a specific type of AI designed to act autonomously within an environment, making decisions based on its own goals.


Besse highlights the use of video games as training grounds for agents due to their well-defined environments and diverse scenarios. He explores the potential of agents to reach Artificial General Intelligence (AGI) and revolutionize fields like scientific research.


You will learn about

  • The concept of AI agents and their relationship to autonomy.

  • How DeepMind leverages video game environments to train agents.

  • The history of DeepMind's work in AI and games, from DQN to AlphaZero.

  • The challenges and potential of agents achieving AGI.

  • Real-world applications for agents, from self-driving cars to virtual assistants.

Watch it

Thank You.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Advertise