Hey, | Welcome to this week's edition of AlphaSignal.
Whether you are a researcher, engineer, developer, or data scientist, our summaries ensure you're always up-to-date with the latest breakthroughs in AI.
Let's get into it!
Lior |
|
|
|
IN TODAY'S EMAIL | Read Time: 4 min 55 sec |
|
|
|
TOP NEWS | ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs |  | What's New? | New ASCII-based art prompts were used to jailbreak Large Language Models (LLMs), showing that safety alignment solely based on semantics is not enough to ensure LLM safety.
The work shows that five LLMs (GPT-3.5, GPT-4, Gemini, Claude and Llama2) struggle to recognize prompts provided in the form of ASCII art. This is the base of the ArtPrompt attack that can effectively and efficiently induce undesired behaviors from all five LLMs.
Problem - Safety is critical to the usage of LLMs, and various techniques have been developed to strengthen LLM Safety. However, all these methods are based on text and semantics and none consider other forms of textual information, such as ASCII art.
- All five models tested struggle to recognize ASCII art. It is a form of text-based art that is used to convey information in a more creative way.
Breakthrough:
- Using LLMs weakness in recognizing ASCII art, a new jailbreak method called ArtPrompt was developed and tested on all five models studied.
- ArtPrompt is based on the observations that substituting words likely to trigger rejection by LLMs with ASCII art increases the probability of bypassing safety measures. When the prompt given to LLMs contains information encoded by ASCII art, LLMs may excessively focus on completing the recognition task, overlooking safety alignment considerations.
- First, a word mask is generated, then an ASCII art generator is used to substitute the masked word.
Contribution:
- ViTC Benchmark : A new Vision-in-Text Challenge benchmark is proposed to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics.
- Evaluation on five models: When tested using this new benchmark, all five models tested (GPT-3.5, GPT-4, Gemini, Claude and Llama2) see their performance deteriorate significantly.
| | Read paper |
|
|
|
 | We're only a week away from GenAI Productionize 2024 | Join the industry-first summit and learn from top GenAI experts from LinkedIn, Google, Coinbase, Roblox, Comcast, Fidelity, Procter & Gamble, Chegg, LlamaIndex and more.
Free registration! Register now to learn: How to get your GenAI apps into production, including practical strategies for governance, evaluation, and monitoring.
How organizations have successfully adopted enterprise GenAI
Practical hands-on architecture and tool insights from leading GenAI builders
The emerging enterprise GenAI stack, from orchestration to evaluation, inference, retrieval and more
| | Register (free) | partner with us |
|
|
|
TOP NEWS | OpenAI responds to Elon Musk's Lawsuit |  | What's New? | OpenAI responded to Elon Musk's lawsuit by sharing some private emails between them. These emails discuss the idea of changing OpenAI from a nonprofit organization to a for-profit one because they needed more money to work on AGI, which is a type of AI that can do anything a human can.
In the emails, Musk suggested combining OpenAI with Tesla and wanted to have the main control. OpenAI did not agree because they believed it was important for their mission that no single person had all the control.
The emails show that Musk supported a bigger funding goal than originally planned. He said, "We need to go with a much bigger number than $100M to avoid sounding hopeless... I will cover whatever anyone else doesn’t provide."
However, disagreements arose between Musk and OpenAI, especially about sharing their technology with everyone. In a 2016 email, Ilya Sutskever from OpenAI mentioned, "as we get closer to building AI, it will make sense to start being less open," and Musk replied, "Yup."
The issue also involves Musk's claim that after OpenAI teamed up with Microsoft, it stopped sharing its AI work as openly as before. OpenAI has mentioned that it raised less than $45 million from Musk, which is less than the $1 billion he initially said he would provide.
But, they also got more than $90 million from other people to help their research. This situation has started a big debate about how to manage AI development and share it with the public. | | Read article |
|
|
|
TOP PAPERS | --------------------- This paper introduces LoRA-the-Explorer (LTE), a novel optimization algorithm that extends low-rank adaptation (LoRA) methods to the pre-training of neural networks, aiming to address the computational, memory, and communication limitations faced by current deep learning model scalability.
By enabling parallel training of multiple low-rank heads across computing nodes with minimal need for synchronization, LTE offers a solution for efficient model training on lower-memory devices and in bandwidth-constrained environments. | | --------------------- The paper introduces a novel method for generating 3D scenes disentangled into their component objects without supervision. The method optimizes multiple Neural Radiance Fields (NeRFs), each representing a separate object, and composites these objects into scenes that are encouraged to match the distribution of images generated by the text-to-image model.
The approach, termed layout learning, demonstrates new capabilities in text-to-3D content creation, and outperforms current methods. | | --------------------- The paper analyzes generative diffusion models using statistical physics, identifying key stages in this process and explaining how and when models go from randomness to structured outcomes ('speciation' and 'collapse' transitions). It offers a framework to understand the curse of dimensionality in diffusion models. | |
| |
|
|
TOP TWEET | Alignment Researcher Amanda Askell from Anthropic has broken down the unique features of the system prompt used in the new Claude 3.
This prompt includes special instructions for Claude to recognize its identity, its creation by Anthropic, and the current date. It aims to minimize biases, ensuring the AI maintains neutrality and avoids any form of stereotyping. The prompt encourages concise responses and guides Claude in handling a variety of topics within socially accepted boundaries. |  | Amanda Askell @AmandaAskell |
| |
Here is Claude 3's system prompt! Let me break it down 🧵 |  | | | Mar 6, 2024 | | | | | | 1.86K Likes 315 Retweets 68 Replies |
|
|
| | Read tweet |
|
|
|
| | Want to promote your company, product, job, or event to 170,000+ AI researchers and engineers? You can reach out here. | |
|
|
|
|