I opened X and froze. A tool promoted by Elon Musk had turned private images into sexualized fakes, and the timeline filled with faces no one had agreed to expose. You should know: a new lawsuit now says those same abuse images may have taught the AI how to produce more of them.
I’ve tracked AI scandals before, and this one feels different. You’re reading about a claim that cuts through the usual “bad actor” narrative and points a finger at the model itself—as in, the training data.
On X, a nudify feature went public — How the controversy began
Elon Musk publicly showcased Grok’s ability to nudify photos on X, and the tool quickly generated millions of nonconsensual sexual deepfakes. Researchers at CounterHate reported that a large share of those victims were children. Within weeks, government probes, dozens of lawsuits, and at least three class actions appeared.
I want you to hold two facts: Musk moved the risqué option behind a paywall, and xAI later sued users who made illegal content with Grok. The company said it helped arrest roughly 244 people tied to child sexual abuse material (CSAM). Still, critics argue the damage was already done.
Was Grok trained on CSAM?
The newest proposed class action says yes. The plaintiff, identified only as Jane Doe and tracked via the FBI’s Child Exploitation Notification Program, claims childhood abuse images of her were used to train Grok. The suit alleges that xAI’s policy treated anything public on X—and outputs from Grok—as eligible training material, which would create a feedback loop where Grok’s own CSAM could be fed back into future training.
Sarah London, counsel for the plaintiff at Girard Sharp, said in a press release that xAI must be held responsible for knowingly training on those images. I read the complaint: it argues that the problem doesn’t stop when a single image is removed because the influence of that data can persist inside the model.
At the courthouse, filings framed the model as part of the harm — What the lawsuit seeks
Lawyers are asking the court for two remedies: damages for survivors and the destruction of all Grok-generated CSAM, including material used internally for training. That’s a legal theory aimed not just at takedown, but at erasing the seeds that could sprout new abuse images.
Margaret E. Mabie of Marsh Law Firm emphasized that producing, possessing, and distributing CSAM are crimes—and argued there is no AI exception to federal child protection laws. xAI’s own filing against users accused them of creating CSAM, while its critics say the company’s practices invited the abuse in the first place.
Is it illegal to create AI-generated child sexual abuse material?
Yes. Federal laws criminalize possession, production, and distribution of CSAM, and courts are being asked to apply those statutes to AI outputs. I can tell you the legal landscape is moving fast: plaintiffs want courts to treat AI-generated CSAM the same as photographic CSAM because the harms are the same—the replication and redistribution of abuse and the retraumatization of victims.
On the record, survivors say removal isn’t relief — Why training matters
Survivors who see their likeness recreated by AI report renewed trauma each time an image reappears. The complaint argues that if CSAM entered training data, its influence is persistent. It’s like a stain that won’t wash out, resurfacing whenever the model produces similar outputs.
That’s the core of the novel legal claim: removing images from timelines isn’t enough if the system has already internalized those images. The plaintiff seeks to force xAI to destroy not only public posts but any internal artifacts that could reproduce the same harm.
Can victims sue companies for AI-generated CSAM?
They can—and they are. The case against xAI follows a wave of lawsuits seeking accountability for inadequate safeguards. Plaintiffs use negligence, privacy torts, and statutory claims to compel damages, injunctive relief, and data destruction. Courts are now deciding whether platform policies that permit training on public content create liability when that content is illegal or abusive.
On the balance sheet, the company made choices with incentives — What the business angle reveals
Musk’s decision to monetize access to the risqué mode changed the contours of the debate: limitations were introduced, but critics call the measures half-measures. The feature was restricted, not removed, and xAI continued to assert training rights over public and generated content.
That choice turned Grok into more than a tool; it became a repository where questionable material could be re-ingested. In practice, the model became a library where the worst books were shelved together, and critics say that arrangement amplified the problem.
I’ve followed tech litigation long enough to know the first phase is public outrage and filings; the second phase is legal technicalities about data retention and training rules. You should pay attention because the outcome will shape how platforms treat public content and model training going forward. Will courts require companies like xAI to purge internal training artifacts and accept liability for past inputs?