Skip to main content
Back

GoogleTakes Over Open Models

TRY GEMMA

February 21, 2024 4 min read

Hey,

Welcome to this week's edition of AlphaSignal. 


Whether you are a researcher, engineer, developer, or data scientist, our summaries ensure you're always up-to-date with the latest breakthroughs in AI. 


Let's get into it!


Lior

IN TODAY'S EMAIL

  • Google's New Open LLM: Gemma

  • Karpathy 2-hour lecture

  • Deep Dive: What is "Needle in a Haystack"?

Read Time: 4 min 01 sec

NEWS #1

Google Releases Gemma: The Best Open LLM Yet

What's New?

Google just unveiled Gemma, a family of lightweight, state-of-the art open models built from the same research and technology used to create the Gemini models.


Key Features of Gemma:

  • Model Variants: Gemma comes in 2B and 7B parameter versions, each available in Instruction and base (non-multimodal) formats.

  • Commercial Use: Fully authorized for commercial applications, opening up new possibilities for businesses.

  • Context Window: Both models support a 8192 token context window.

  • Performance: The 7B model notably outperforms competitors such as Mistral AI 7B and LLaMa 2 in Human Eval and MMLU tests, scoring 64.56 on MMLU.

The architecture of Gemma incorporates several significant advancements over traditional transformer models. Innovations include:

  • Multi-Query Attention and RoPE: Enhances attention mechanisms and positional encoding for more effective learning.

  • GeGLU Activations: Replaces the standard ReLU for more dynamic model responses.

  • Normalization Techniques: Utilizes RMSNorm for better stability across layers.

Google has made Gemma accessible on platforms like Hugging Face, Kaggle, and Vertex AI.

> Technical Report Here
> Official Page Here

Try gemma

Explain And Document The Most Complex Tasks In Seconds With AI

It’s time to delegate that work to AI. guidde is a GPT-powered tool that helps you explain the most complex tasks in seconds with AI generated documentation.

  • Turn boring documentation into stunning visual guides

  • Save valuable time by creating video documentation 11x faster

  • Share or embed your guide anywhere for your team to see

Simply click capture on our browser extension and the app will automatically generate step-by-step video guides complete with visuals, voiceover and call to actions.

The best part? Our extension is 100% free. Try it here →

Try it here

partner with us

NEWS #2

Karpathy Uploads a 2-Hour Long Tutorial

Andrej Karpathy just uploaded a new 2 hour-long tutorial on how to build a tokenizer similar to those used in OpenAI's GPT from the ground up. 


The session is not just theoretical; it provides practical coding exercises, including how to implement the encode() and decode() functions, and explores the broader implications of tokenization on LLM performance.

The tutorial covers:

  • Tokenizer's role: encoding strings, decoding tokens, and training with Byte Pair Encoding (BPE).

  • Implementation details: counting pairs, merging, and vocabulary management.

  • Tokenization issues: quirks, GPT-2/4 regex differences, and prompt compression impact.

  • Alternatives: SentencePiece library and multimodal tokenization with vector quantization.
Watch now

DEEP DIVE

"Needle in a Haystack"

"Needle in a Haystack" testing is an approach for evaluating the in-context retrieval performance of large language models (LLMs). This benchmark tests the models' limits in recalling fine details from extensive inputs, which will be more and more needed as the context length of models increases.


How does it work?

  • a 'needle' — a fact or statement — is embedded within a 'haystack' — a lengthy, detailed context.

  • the model is tasked to retrieve this 'needle' from its memory.

  • model's retrieval precision is measured by varying the the position of the 'needle' and the size of the 'haystack'

This technique was used to test various models from OpenAI, Anthropic and more recently Google. 


How can you use it?

The Github repository that first introduced this testing framework provides a structured way to implement these tests, with parameters that can be adjusted to suit different testing needs. It allows for the saving and analysis of results, crucial for iterative improvement.


By understanding these patterns, you can fine-tune your models, adjust training data, or even redesign aspects of the model architecture to enhance in-context retrieval abilities. 


Is this a good test for retrieval performance?

The 'Needle in a Haystack' test is a useful framework to assess a model’s retrieval capabilities, however, it has its limitations. 


  • models should be able to find multiple needles in a haystack, and should be evaluated on the number of correct needles retrieved instead of a unique one

  • the performance of models in this test is affected by the prompt and the nature of the needle

  • A more robust testing framework could be to ask a question that requires finding and reasoning on multiple needles in a haystack. Such testing frameworks still need to be created and standardized. 


The Needle in a haystack framework is only one testing framework that gauges the performance of models in retrieval, but there are many other benchmarks that are also important, like the in-context learning capabilities.

Gemini Pro 1.5 seems to be a good in-context learner as it was able to learn how to speak a new language from extensive documents on its grammar and vocabulary. 

Not Great   Good    Amazing

Thank You.

Want to promote your company, product, job, or event to 170,000+ AI researchers and engineers? You can reach out here.