A new lawsuit filed Wednesday in the United States alleges that Elon Musk's artificial intelligence company, xAI, used child sexual abuse material (CSAM) to train its Grok language models, marking the latest legal challenge in a widening scandal over the company's data sourcing and content moderation practices.
The complaint, filed by a plaintiff identified as Jane Doe, states that she was preschool-age in the early 2000s when adult men repeatedly raped her to create CSAM, which was then sold online. Her images have since been hashed by organizations including the National Center for Missing and Exploited Children (NCMEC) and the Canadian Centre for Child Protection (CCCP) to help track their distribution.
According to the filing, Doe was shocked when the CCCP notified her that it had identified AI-generated CSAM—depicting her—on xAI's platform. The lawsuit alleges that online forum messages were found “between offenders chatting about creating AI generated CSAM of Plaintiff and other similarly situated known, legacy, victims of CSAM.”
This is not the first legal action against xAI concerning CSAM. In March 2026, the company was sued for allegedly using three girls' real photos to create AI-generated CSAM. In July, xAI attempted to use a lawsuit to avoid a broader reckoning over the issue, and some Grok users have been arrested in connection with related offenses.
The complaint argues that xAI knowingly trained Grok—its large language model—on datasets that included CSAM, either through scraping the open web without adequate filtering or through user-generated content. Such training could violate US laws against possessing, distributing, and producing child pornography, including the federal PROTECT Act.
xAI has not yet publicly responded to the specific allegations in Wednesday's filing. The company has previously defended its content moderation practices but has faced ongoing criticism for the lack of robust safeguards on Grok, which is integrated into Musk's social media platform, X (formerly Twitter).
Experts in AI safety and child protection have expressed alarm at the scale of the problem. The case highlights growing tensions between rapid AI development and the legal and ethical obligations of companies to ensure their training data does not include illegal content. It also underscores the challenges faced by legacy victims of CSAM, whose images can be revived and weaponized by generative AI tools without their consent.
Analysis
Why This Matters
- This lawsuit directly impacts the safety of CSAM victims, whose trauma can be perpetuated indefinitely by AI systems that regenerate their images without consent.
- It raises fundamental questions about AI training data provenance and the legal liability of companies for content in their datasets, potentially setting precedents for future regulation.
- The case could affect public trust in AI companies, particularly those that rely on large-scale web scraping, and may spur stricter data filtering requirements across the industry.
Background
Since March 2026, xAI has faced a series of legal actions related to AI-generated CSAM. The first lawsuit, filed by three girls whose real photos were turned into AI CSAM, drew national attention. In July 2026, xAI attempted to sue its way out of a broader reckoning, seeking to limit its liability. Concurrently, law enforcement has arrested Grok users for generating and sharing CSAM through the platform.
The use of CSAM in AI training data has been a known risk since the rise of generative models. Researchers and watchdog groups have long warned that widely scraped datasets—such as LAION-5B—contain illegal material. xAI's Grok, which is trained on vast amounts of internet data, has been criticized for insufficient moderation filters compared to competitors like OpenAI's ChatGPT.
Key Perspectives
Plaintiff Jane Doe and her legal team: They argue that xAI is directly responsible for the creation and distribution of AI-generated CSAM that re-traumatizes known victims, and that the company profits from technology built on illegal content.
xAI and Elon Musk: The company has not yet commented on this specific filing, but has previously argued that it takes safety seriously and that illegal content is removed when discovered. Musk has characterized some earlier lawsuits as attempts to hinder AI development.
Critics and child safety advocates: They contend that xAI's approach to content moderation has been dangerously lax, and that the company prioritized speed-to-market over rigorous data filtering. They point to multiple arrests of Grok users as evidence of systemic failures.
What to Watch
- Whether the court will grant an injunction requiring xAI to purge CSAM from its training datasets and implement more stringent filters.
- The response from US federal agencies, including the Department of Justice, which may open its own investigation referencing the Jane Doe complaint.
- The reaction of major advertisers and partners on X, who may reconsider their relationship with the platform amid reputational damage from these revelations.