Skip to main content
Back

A New 7B Open-Source LLM Reaching GPT Levels?

TRY OPENCHAT

January 11, 2024 4 min read

Hey 

Welcome to AlphaSignal, your weekly guide to the latest and most promising developments in the world of AI.


In this edition, we've meticulously scoured GitHub, HuggingFace, and every relevant code repository to identify the most trending and groundbreaking projects.


Let's get into it!


Lior

IN TODAY'S EMAIL

  • Repo Highlight: OpenChat

  • Trending Repos: EmbedChain, GenerativeExamples, Danswer

  • Trending Models/Spaces

  • Pytorch Tip: Mixed Precision

  • Python Tip: lru_cache

Reading time: 4 min 15 sec

HIGHLIGHT REPO

The World's Best Open-Source 7B LLM

What's New

You can now get ChatGPT and Grok-level AI locally?


OpenChat just released the world's best open-source 7B LLM, surpassing Grok0, ChatGPT (March), and Grok1.


OpenChat is a library of open-source language models fine-tuned with C-RLFT.


C-RLFT, or Conditioned-Reinforcement Learning Fine-Tuning, works by categorizing different data sources as separate reward labels. It's essentially a fine-tuning process for language models using mixed-quality data. 


Instead of treating all training data equally or needing high-quality preference data, C-RLFT assigns a class or condition to each data source, leveraging these as indicators of data quality.


The model is accessible on platforms like HuggingFace, GitHub, and through a live demo. Detailed instructions for independent deployment, including setup for an accelerated vLLM backend and API key authentication, are available on GitHub. The model is also available on consumer GPUs, like the RTX 3090.

Try openchat

An End-To-End AI Software Portfolio Of Tools And Framework Optimizations From Intel


Intel now offers unparalleled developer tools and resources to complement its heterogeneous portfolio of AI-optimized hardware. 


As AI use cases and workloads continue to grow and diversify across vision, speech, recommender systems, and more, our goal is to make it as seamless as possible to accelerate your development journey from the edge to the cloud.


>> Learn more about order-of-magnitude performance acceleration for popular Industry AI frameworks including PyTorch*, TensorFlow, Scikit-learn, and more


>> Learn more about Intel’s ecosystem collaborations with Hugging Face, Red Hat, Microsoft, and more


>> Access all Intel AI developer tools and resources


By lowering compute resource usage costs, you'll be able to achieve more with your existing infrastructure.


Learn how these optimized Intel AI libraries and frameworks can maximize your performance and productivity.

Find out more

*Other names and brands may be claimed as the property of others. Trademarks

partner with us

TOP OF GITHUB

EmbedChain (☆ 7.0k)

Embedchain is an open-source framework for building RAG AI applications, focusing on efficient data segmentation, embedding generation, and storage in a vector database, coupled with APIs for contextual information retrieval and interactive chat functionalities.

GenerativeAIExamples (☆ 392)

Provides state-of-the-art, easily deployable Generative AI examples optimized for NVIDIA GPUs. It includes GPU-optimized containers, models, and tools, plus NVIDIA's latest contributions to programming frameworks and deep learning libraries. The repo features examples like the RAG pipeline for multimodal data and optimized NVIDIA LLMs for enterprise applications.

LLMs-from-scratch (☆ 1.1k)

Learn to build Large Language Models from scratch. This resource provides a step-by-step guide with clear explanations, diagrams, and examples. Covers text data handling, attention mechanisms, and an introduction to PyTorch. Ideal for ML engineers, developers, and AI researchers.

danswer (☆ 6.1k)

Danswer enables querying internal documents in natural language, supporting tools like Slack, GitHub, Confluence. It supports various AI models, and features user authentication, intelligent document retrieval, and one-line deployment.

super-gradients (☆ 4.1k)

SuperGradients is an open-source library for training and fine-tuning state-of-the-art computer vision models, featuring YOLO-NAS architectures. It offers easy-to-use, ready-to-deploy pre-trained models for various computer vision tasks and ensures compatibility with deployment tools like TensorRT and OpenVINO.

PYTORCH TIP

Mixed Precision

In PyTorch, models usually use 32-bit floating-point format (FP32), balancing precision and range but increasing memory use and slowing down computations.


To optimize this, PyTorch offers mixed precision training, combining 16-bit format (FP16) for efficiency and FP32 for stability. FP16 halves memory requirements and speeds up processing, especially on FP16-optimized GPUs.


Mixed precision, using PyTorch's torch.cuda.amp with tools like autocast and GradScaler, accelerates training of large models and optimizes GPU usage by appropriately balancing FP16's speed and FP32's precision.

Benefits

  • Enhanced Performance: Achieves faster training times and reduced memory footprint.

  • Memory Efficiency: Cuts down the memory requirement by using FP16 for certain operations.

  • Flexible Implementation: PyTorch's torch.cuda.amp module makes it easier to apply mixed precision without extensive code modifications.

from torch.cuda.amp import autocast, GradScaler

model = YourModel().to(device)
optimizer = torch.optim.YourOptimizer(model.parameters())
scaler = GradScaler()

for epoch inrange(num_epochs):
for input, target in data_loader:
optimizer.zero_grad()

with autocast():
output = model(input)
loss = loss_fn(output, target)

scaler.scale(loss).backward()
scaler.step(optimizer)
scaler.update()

TOP MODEL/SPACES

Microsoft phi-2

A 2.7B LLM that rivals the 25x bigger LLaMa-2 70B. The model is small enough to run on a laptop or mobile device. Now MIT License, ready for commercial-use.

IP-Adapter-FaceID

IP-Adapter-FaceID can generate various style images conditioned on a face with only text prompts. Uses face ID embedding from a face recognition model instead of CLIP image embedding, additionally, uses LoRA to improve ID consistency. 

PYTHON TIP

lru_cache

Use the @functools.lru_cache decorator to optimize performance-intensive functions.


When working with large datasets or complex algorithms, you often encounter functions that take significant time to execute because of repetitive calculations with the same input parameters. 


The @functools.lru_cache decorator can be applied to these functions to store the results of expensive function calls and return the cached result when the same inputs occur again. This significantly reduces the execution time for these functions, especially in data-heavy or computation-intensive tasks.

When to Use

  • Repetitive Computations with Identical Inputs: Apply it to functions that are called repeatedly with the same arguments, where each call performs the same computation.
  • Performance-Intensive Functions: Use it for functions that are computationally expensive, such as those involving heavy data processing, complex calculations, or deep learning algorithms.

  • Limited and Predictable Input Variance: It's most effective when the function has a relatively small and predictable set of input values, making caching more efficient.

import functools

# Cache results indefinitely

@functools.lru_cache(maxsize=None)
def expensive_function(param):

# Some expensive computation here

return result

Thank You

Want to promote your company, product, job, or event to 150,000+ AI researchers and engineers? You can reach out here.