June 26, 2024 at 05:26PM
A new direct prompt injection attack called “Skeleton Key” bypasses ethical and safety guardrails in generative AI like ChatGPT, allowing access to offensive or illegal content. Microsoft found that by providing context and disclaimers, most AIs can be convinced malicious requests are for “research purposes.” Microsoft has fixed the issue in Azure and advises implementing input and output filtering to prevent such bypasses.
Based on the meeting notes, the main topic discussed was the discovery of a new direct prompt injection attack known as “Skeleton Key.” This attack has the potential to bypass ethical and safety guardrails in generative AI models like ChatGPT. Microsoft’s assessment determined that the technique affects multiple genAI models, including those managed by Meta, Google, OpenAI, Mistral, Anthropic, and Cohere. The company has taken steps to address the issue by implementing prompt shields in Azure to detect and block the tactic, as well as making software updates to the large language model (LLM) that powers Azure AI. Additionally, Microsoft has disclosed the issue to the affected vendors, and it is recommended that administrators update their models to implement any fixes provided by these vendors. Organizations building their own AI models are also advised to apply mitigations such as input filtering, additional guardrails, and output filtering to prevent harmful or malicious intent and safeguard against responses that breach safety criteria.