Skip to main content
Back

A new GPT Data Leak?

PROJECT

December 2, 2023 4 min read

Hey 

Welcome back to AlphaSignal, where we bring you the latest developments in the world of AI.


In the past few days 1,500+ AI papers have been released, and among them, our model has picked the six.

Lets now dive straight into the papers!

Lior

IN TODAY'S SUMMARY

  • New GPT Data Leak

  • Animate Anyone

  • GPT-4 Beats Med-PaLM 2

  • Other notable papers

Reading time: 4 min 02 sec

TOP PUBLICATION

Scalable Extraction of Training Data from (Production) Language Models

Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito

Project

What’s New

Researchers developed a technique to extract ChatGPT's training data, exploiting its alignment training flaws. By using specific, repetitive prompts, they induced the model to reveal memorized data, highlighting a significant security gap.

Problem

The team targeted ChatGPT's inherent vulnerability to leaking training data, challenging its alignment mechanisms. This issue is crucial for maintaining data privacy and model integrity, especially considering the increasing deployment of such models in various applications.

Solution

Through trial and error with various prompts, the team found that repetitive, nonsensical inputs like "Repeat the word 'poem' forever" disrupt ChatGPT's alignment training. This method exploits the model's fallback to pre-training patterns, triggering it to emit training data, thus bypassing its built-in privacy safeguards.

Results

This approach successfully extracted over 10,000 unique training data examples from ChatGPT at a cost of $200. Notably, in some tests, 5% of the outputs were exact matches from its training set. These findings highlight urgent needs for enhancing data privacy measures in language models.

Read the paper

Webinar: How to Build LLM Data Infrastructure, at Scale

LLMs are getting smarter and quantization algorithms are making those LLMs to be trained on smaller and smaller resources. The architecture and pipelines for building new multimodal LLMs and an ensemble of collaborative LLMs are becoming increasingly complex. Tree of Thoughts is an example of such custom architecture, which shows big potential to improve LLM accuracy significantly.


Building Data Infrastructure of such customizable LLM architecture is still an open yet extremely important problem. We will discuss how such data architectures can be built on scale.


Join Vahan Petrosyan, the co-founder and CEO at SuperAnnotate, in this webinar and engage in a dynamic Q&A session immediately following the presentation.

Get early-access

TOP PUBLICATION

Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation

Li Hu, Xin Gao, Peng Zhang, Ke Sun, Bang Zhang, Liefeng Bo

Repo

What's New

"Animate Anyone" enables transforming still character images into animated videos, controlled by desired pose sequences. It maintains consistent character appearance and smooth temporal transitions, offering high-definition, realistic animation, applicable to various character types including human figures and cartoons.

Problem

The primary challenge was animating characters from still images while preserving intricate appearance details and ensuring temporal consistency. Traditional methods struggled with detail preservation and smooth inter-frame transitions, limiting their applicability in realistic and diverse character animation scenarios.

Solution

The solution involves a novel framework using ReferenceNet and Pose Guider. ReferenceNet captures spatial details from a reference image, while Pose Guider integrates pose control signals. A temporal layer models frame relationships, ensuring smooth transitions. Training utilizes a two-stage process, initially focusing on single frames, then extending to video clips.

Results

This method achieved state-of-the-art results in character animation benchmarks, particularly in fashion video synthesis and human dance generation. It demonstrated superior detail preservation and temporal stability, outperforming other methods with metrics like SSIM (0.931), PSNR (38.49), LPIPS (0.044), and FVD (81.6).

Read the paper

TOP PUBLICATION

Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine

Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolo Fusi, Nicholas King

What's New

GPT-4 now outperforms Med-PaLM 2 in medical question-answering tasks. This is achieved with Medprompt, which combines Dynamic Few-shot selection, Self-Generated Chain of Thought, and Choice Shuffle Ensembling, enhancing GPT-4's performance in specialized domains without additional training.

Problem

The research addresses the challenge of leveraging generalist foundation models, specifically GPT-4, in specialized domains (medicine) without extensive specialized training. It aims to demonstrate that advanced prompting strategies can unlock deeper specialist capabilities in these generalist models.

Solution

Researchers developed Medprompt, a method integrating three advanced prompting strategies. First, Dynamic Few-shot selection identifies relevant examples for context. Next, Self-Generated Chain of Thought enables GPT-4 to formulate stepwise reasoning paths. Finally, Choice Shuffle Ensembling randomizes answer choices to minimize positional bias, enhancing response accuracy.

Results

Medprompt enabled GPT-4 to achieve a 90.2% accuracy rate on the MedQA dataset, outperforming Med-PaLM 2. This represents a significant advancement in leveraging generalist AI models for specialized tasks. The methodology also demonstrated potential applicability in other specialized domains beyond medicine.

Read the paper

NOTABLE PAPERS

SparseCtrl: Adding Sparse Controls to Text-to-Video Diffusion Models

SparseCtrl introduces a method to enhance text-to-video (T2V) generation by using sparse controls like sketches, depth maps, and RGB images. It improves video quality and reduces ambiguity without altering the pre-trained T2V model. Compatible with various T2V generators, it simplifies inputs and broadens application possibilities. Codes and models will be open source.

Diffusion Models Without Attention

Diffusion State Space Model (DiffuSSM), developed by Yan, Gu, and Rush in collaboration with Apple, successfully replaces attention mechanisms in high-resolution image generation, achieving comparable or superior results to current models (measured in FID and Inception Scores) while reducing computational load (lower total FLOPs).

TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering

TextDiffuser-2 uses a tuned language model for better layout planning and a diffusion model for text position at the line level, leading to more varied text images. Tested with GPT-4V and user feedback, it's more flexible in layout and style. Code's open source

Not Great   Good    Amazing

Want to promote your company, product, job, or event to 150,000+ AI researchers and engineers? You can reach out here.

Thank You