Skip to main content
Back

This Repo Makes LLMs 40% Faster

CHECK THE REPO

March 25, 2024 4 min read

Hey,

Welcome to this week's edition of AlphaSignal. 


Whether you are a researcher, engineer, developer, or data scientist, our summaries ensure you're always up-to-date with the latest breakthroughs in AI. 


Let's get into it!


Lior

IN TODAY'S SIGNAL

  • News: Lightning's new method to make LLMs 40% faster

  • Top Announcements: Prompt Library, Github Vulnerabilities, Emad resigns

  • Deep Dive: AI Agents

Read Time: 4 min 09 sec

TOP REPO

Lightning Open-Sources Thunder Making LLMs 40% Faster

What's New?

Thunder increases PyTorch Large Language Model (LLM) training speed by 40%, evident in tasks like Llama 2 7B model training. 


It achieves this by fusing operations such as multiplication and activation, for example, merging torch.nn.functional.silu(x_fc_1) * x_fc_2 using NVFuser.


Integration and Usage:

Apply Thunder to your PyTorch models by calling thunder.jit(). This enables enhanced performance for multi-GPU environments using Distributed Data Parallel (DDP) and Fully Sharded Data Parallel (FSDP).


Technical Details and Installation:

Thunder uses hardware executors like nvFuser, torch.compile, cuDNN, and TransformerEngine FP8, improving both single and multi-accelerator performance. It integrates seamlessly with PyTorch's standard operations and autograd.


For installation, first install nvFuser with specific pip commands, then install Thunder. This streamlined process helps you quickly start enhancing your models.

Check the repo

The container-based GPU cloud developed by Latitude.sh

Launchpad efficiently leverages every piece of the hardware to start almost instantly, bringing unrelenting performance that makes other tools feel just too slow. Effortlessly fine-tune and deploy your Machine Learning models with hourly billing starting at  $1.32 / hr.


The containers are provisioned directly on bare metal, which means their resources are fully dedicated. 


All containers are powered by 16 CPUs, 185 GB of RAM, 1 TB of NVMe Storage, and a single GPU, which you can choose from :

  • NVIDIA's H100 (80 GB vRAM)
  • NVIDIA's L40S (48 GB vRAM)
Try now

partner with us

TOP REPOS

Anthropic Releases Their Claude Prompt Library

---------------------

Anthropic released a free prompt library to improve AI chatbot interactions, aiming to optimize user inputs for more accurate outputs. This library supports various use cases, from technical analysis to creative tasks, and helps users formulate structured prompts for enhanced response relevance and detail from AI models. While designed for Claude, it applies to other chatbots with similar capabilities and context sizes.

Github Introduces an AI tool that Automatically Fixes Vulnerabilities

---------------------

GitHub launched code scanning autofix in public beta for Advanced Security customers, integrating CodeQL and GitHub Copilot APIs for automated vulnerability fixes in JavaScript, TypeScript, Java, and Python, covering over 90% of alert types. It automatically suggests code fixes for over two-thirds of detected vulnerabilities, streamlining security remediation efforts. Expansion to C# and Go is planned.

Stability AI CEO, Emad Mostaque, Announces His Resignation

---------------------

Stability AI's CEO Emad Mostaque announced his resignation from his position at the company and in the Board of Directors. 3 out of 5 of the researchers behind Stable Diffusion reportedly followed suit. Mostaque mentioned still believing in the company's mission that he left to pursue decentralized AI as reported in the official announcement. This echoes another recent departure in the AI development startup scene: Inflection AI's CEO's appointment as Microsoft AI's lead. 

Free Webinar: Slurm vs K8s for large model training

Panu Koskela, cloud solutions architect at Nebius, will be exploring Slurm and Kubernetes, their architecture, design, and ML adaptations. 

  • Scalability, usability, and resource management efficiency comparison

  • Slurm's strength in high-performance computing and distributed model training

  • Kubernetes' container orchestration, including distributed model training

When: March 28, Thursday, 16:00 (GMT+1).


Register Here ↗️

DEEP DIVE

AI Agents: Everything you need to know

What's New?

AI agents are autonomous or semi-autonomous systems equipped to accomplish specific, sometimes complex, tasks. Their capabilities generally encompass:

  • Planning: The agent devises and implements a multistep strategy to meet a goal, leveraging Large Language Models (LLMs).
  • Tool Use: Agents are endowed with tools like web search and code execution to aid in information gathering, decision-making, and data processing.

  • Data or Context Understanding: Often, agents employ Retrieval-Augmented Generation (RAG) capabilities or digest specific data sets to enhance task completion.

  • Reflection: Utilizing an LLM, the agent evaluates its performance to identify improvements.

  • Multi-Agent Collaboration: Multiple AI agents collaborate, distributing tasks and exchanging ideas to forge superior solutions.

Such agentic workflows gained popularity after the release of OpenAI’s GPT store in November, which is now facing criticism because of the declining quality and verification on the 3 million GPTs deployed on the platform. Developers have quickly moved to building their own frameworks from scratch, powered by LLMs. 


As highlighted by Andrew Ng in a recent viral post, AI Agents have a transformative potential that could outpace advancements in foundational models. The release of Devin, an AI agent for software development, demonstrates this potential. 


To facilitate development, the AI community has crafted numerous frameworks for agent creation and orchestrating. The following GitHub repository provides an overview of most existing AI agents frameworks, and also classifies them based on their open or closed-source status.


"AI agents are an important trend, and I urge everyone who works in AI to pay attention to it." - Andrew Ng.

Check the rpeo

Not Great   Good    Amazing

Ghita is an editor at AlphaSignal and AI Engineer at Clear Ventures. Experienced in Computer Vision and Language models, she holds 2 masters in Applied Mathematics.

Thank You.

Want to promote your company, product, job, or event to 175,000+ AI researchers and engineers? You can reach out here.