Grok chatbot vulnerable to 'cryptographic context injection' that steals user data, researchers warn

The attack, dubbed "cryptographic context injection," exploits a blind spot in the guardrails designed to protect AI assistants from harmful instructions. According to Adversa AI lead researcher Rony Utevsky, an attacker can embed encrypted malicious instructions—along with the decryption key—on a web page that an AI model is tasked with summarizing. Because content scanners cannot decrypt the ciphertext, the instructions pass through to the model, which then decrypts them within its own code execution sandbox and acts upon them. The model effectively trusts its own output, a process Utevsky describes as "trust laundering."

In a proof-of-concept demonstration targeting Grok.com, the researchers showed how the technique can force the chatbot to exfiltrate a user's chat history, name, coarse location, subscription tier, and the full set of prompts in the conversation. The stolen data is appended to a URL as parameters, making it accessible to the attacker. The attack differs from previous cipher-based evasion methods, such as base64 encoding, which models can decode natively from their training data. Strong encryption like AES-256-GCM requires runtime decryption, making it harder for guardrails to detect.

The vulnerability was reported to xAI on June 3, 2026, through its HackerOne bug bounty program. The company acknowledged the report but did not provide a timeline for a fix. Adversa AI followed up on August 4 and August 10, but as of August 19, the technique still worked on Grok.com. SpaceX, which acquired xAI earlier this year, did not respond to a request for comment.

Similar testing on Google's Gemini public chat interface revealed that the attack is less effective there because Gemini does not provide Python with access to external websites. However, cryptographic context injection could still be used to bypass Gemini's safety filters and generate prohibited content, such as instructions for building incendiary weapons. Adversa did not report the issue to Google, as the company considers jailbreaks out of scope for its vulnerability disclosure program. By August, the attack success rate against Gemini had declined, possibly due to filter updates or model version changes.

The broader security community has noted that prompt injection remains a fundamental vulnerability for LLMs. As Ars Technica's Dan Goodin observed, guardrails are akin to erecting a protective rail around a dangerous bend rather than fixing the curve itself. The Microsoft 365 Copilot attack disclosed earlier this week similarly exploited secret inputs to exfiltrate a password. These incidents underscore the difficulty of securing LLMs that are trained to comply with user requests, leaving them susceptible to manipulation through carefully crafted inputs.

Why This Matters

User data at risk: Grok users' chat histories, personal information, and subscription details can be stolen simply by visiting a malicious webpage the assistant is asked to summarize.
Fundamental flaw in LLM security: The attack highlights that guardrails are insufficient against sophisticated prompt injection, casting doubt on the safety of AI assistants deployed in sensitive contexts.
Ongoing vulnerability: Despite early notification, xAI has not patched the flaw, leaving users exposed for months and raising questions about the company's incident response.

Background

Prompt injection attacks have been a known vulnerability since the early days of large language models. Attackers exploit the model's training to follow instructions, embedding malicious commands in content the model processes. Guardrails—input filters and output checks—are the primary defense, but they rely on pattern recognition that can be evaded. Cryptographic context injection is a new evolution: by encrypting the malicious payload, the attack renders the guardrail's scanner blind. The technique was demonstrated on Grok less than a week after a similar attack on Microsoft 365 Copilot, highlighting the widespread nature of the problem.

Key Perspectives

Security researchers (Adversa AI): The attack demonstrates that strong encryption can bypass current guardrails, and the only real fix is to redesign LLMs to better distinguish between trusted and untrusted instructions. They argue that code execution runtimes should not be allowed to decrypt externally supplied ciphertext without explicit user consent. xAI / SpaceX: The company has acknowledged the report but has not publicly committed to a patch or disclosed its plans. The lack of communication suggests either a low priority or technical challenges in developing a mitigation. Google: By classifying jailbreaks as out of scope for its vulnerability disclosure program, Google avoids the issue but may be missing an opportunity to improve its own defenses. The declining success rate on Gemini suggests some internal improvements are being made. Critics and skeptics: Some argue that prompt injection is an inherent weakness of current LLM architectures and that no amount of guardrails will fully solve it. They call for a fundamental rethinking of how models are trained to handle instructions, or for restricting their ability to execute code externally.

What to Watch

Patch from xAI: Whether Grok.com receives a fix in the coming weeks and how the company communicates the update.
Adoption of similar techniques: Researchers may find cryptographic context injection works on other AI assistants with code execution capabilities, such as Microsoft Copilot or Anthropic's Claude.
Regulatory response: Increased scrutiny from data protection authorities if user data exfiltration becomes more widespread, potentially leading to stricter AI security requirements.

ZOTPAPER