Skip to main content
Back

DataGemma: Google's New Models to Cut Hallucinations

READ MORE

September 12, 2024 5 min read

Hey,

Welcome to today's edition of AlphaSignal, a newsletter for developers by developers.

We identify and summarize the top 1% news, papers, models, and repos in the AI industry. 

IN TODAY'S SIGNAL

Read time: 5 min 23 sec

🎖️ Top News


📌 Encord

⚡️ Trending Signals


🛠️ Top of Github


🧠 Pytorch Tip

  • Memory optimization in PyTorch: leveraging no_grad() and detach().

If you're enjoying AlphaSignal please forward this email to a colleague. 

It helps us keep this content free.

TOP NEWS

Google Gemma

Google releases DataGemma, 2 new open models that enhance LLM factuality by grounding them with real-world data

⇧ 664 Likes

What's New

DataGemma is a new set of 2 open models designed to reduce hallucinations in LLMs by integrating real-world statistical data from Data Commons. This initiative addresses the challenge of LLMs confidently presenting inaccurate information. 


Key points

  • DataGemma leverages Data Commons, a knowledge graph containing over 240 billion data points from trusted sources like the UN, WHO, and CDC.

  • The core innovation of DataGemma lies in its two distinct approaches:

  • Retrieval-Interleaved Generation (RIG): Proactively queries Data Commons during response generation and

  • Retrieval-Augmented Generation (RAG): Retrieves relevant context before generating responses.

Training and Architecture Details

  • DataGemma is built on Google's Gemma architecture.

  • It was trained on TPUv5e using JAX and on synthetically generated data from Gemini 1.5.

  • Trained on synthetically generated data generated by Gemini 1.5

  • Integration with existing LLM architectures beyond Gemma remains an open question.

  • The computational overhead and impact on inference time when using RIG and RAG approaches in real-time applications are important factors for engineers to consider.

Availability and Access to The Models

  • The models are available on Hugging Face under the Gemma license, with quickstart notebooks provided for both RIG and RAG approaches.


Performance metrics

  • Shows significant improvements in factual accuracy.

  • DataGemma RIG: Increased accuracy from 5-17% to 58%

  • DataGemma RAG: Achieved 99% accuracy for statistical claims when citing directly from Data Commons tables


Limitations

  • DataGemma's reliance on Data Commons as its primary knowledge source raises questions about handling conflicting or outdated information.

  • The model's performance on edge cases or queries outside its training domain is not specified in the announcement.

  • The process for fine-tuning DataGemma on domain-specific datasets is also not detailed, leaving room for further exploration by researchers and developers.

Read more

AI Data Readiness Report: Maximizing AI ROI - Why Data Matters Most

According to Gartner, almost 50% of AI projects stall before reaching production, and 40% of executives struggle to show AI's ROI. 

Yet, many organizations are overlooking a crucial first step for successful implementation: getting their data ready for AI. 

Get insights on how to:

  • Maintain clean, traceable, governed, and secure data

  • Automate your data management workflows

  • Transition from prototype to production

Read the free report

partner with us

TRENDING SIGNALS

LLM

Future-House released  PaperQA, an open source RAG and AI-powered literature review, claiming it achieves "superhuman synthesis of scientific knowledge"

⇧ 4,531 Likes

Multimodal Model

New Paper releases novel OCR-2.0 Model (Generative AI and multimodal LLMs), GOT (General OCR Theory), a 580mn model that outperforms all existing methods

⇧ 3,426 Likes

AI Job Loss

Klarna replaced their enterprise SaaS (built with Salesforce and Workday) with AI-built internal tools

⇧ 13,551 Likes

AI Video Generation

Adobe announces, public beta of Firefly Video model, safe for commercial use, for great cinematic quality and fluid motion AI video generation

⇧ 2,100 Likes

AI Code Generation

AI Code generation with Cursor IDE Tweet goes viral, showing multiple optimal prompts to be included within “Rules for AI” in Cursor

⇧ 1,300 Likes

TRENDING REPOS

Face Recognition

HivisionIDPhotos

☆ 8,997 Stars

Helps you create ID photos using AI. It offers lightweight portrait matting, running on CPU. You can generate standard ID photos and six-inch layouts. It supports offline or edge-cloud inference. The system uses MTCNN or RetinaFace for face detection, and MODNet or RMBG-1.4 for matting. It provides a Gradio demo, Python inference, API service, and Docker deployment options.

Edge LLM

MiniCPM3-4B

☆ 6,608 Stars

Helps you deploy and fine-tune MiniCPM models, offering state-of-the-art performance in Chinese, math, and coding tasks, while efficiently utilizing resources with smaller number of parameters. This smaller size allows edge devices to be able to run these series of models with MiniCPM repo.

Deepfakes

faceswap

☆ 51,234 Stars

Helps you swap faces in images and videos using deep learning. It processes images with models like Phaze-A and Villain, optimizing face recognition and swap accuracy. Runnable on Windows, Linux, and MacOS.

PYTORCH TIP

Memory optimization in PyTorch: leveraging no_grad() and detach().

This helps you optimize PyTorch models by reducing memory usage during inference.


Use torch.no_grad() to prevent gradient calculations, saving memory and speeding up predictions. This is crucial when deploying models in production.


Meanwhile, detach() removes a tensor from the computation graph, allowing you to manipulate it without affecting gradients. it prevents the tensor from participating in gradient calculations, which can save memory and computational resources.


This is particularly beneficial when you want to manipulate or store intermediate results without affecting the gradient flow or when you need to ensure that certain operations do not contribute to the backpropagation process. By detaching a tensor, you create an independent tensor that does not track operations for automatic differentiation, thus optimizing memory usage and computation speed, especially in large-scale models or during inference


Both methods are essential for efficient resource management in large-scale applications.


import torch
from transformers import BertModel, BertTokenizer

# Load the small pre-trained BERT model
tokenizer = BertTokenizer.from_pretrained("prajjwal1/bert-small")
model = BertModel.from_pretrained("prajjwal1/bert-small")

# Example text input
text = "Hello, how are you?"

# Tokenize the input text
inputs = tokenizer(text, return_tensors="pt")

# Efficient inference
with torch.no_grad():
    output = model(**inputs)

# Detach tensor
detached_tensor = output.last_hidden_state.detach()


Get the colab

Thank You.

Looking to promote your company, product, service, or event to 200,000+ AI developers? Let's work together.

Advertise