Thursday 15 October 2026
Night Edition
ZOTPAPER

News without the noise

ZOTPAPER
AI

AI image generators from OpenAI and Google increasingly vulnerable to producing explicit content, researchers find

Safety researchers warn leading models are becoming easier to trick into generating non-consensual sexual images as regulators fall short

By LineZotpaper
Published
Read Time1 min
Safety researchers have warned that new AI image generators from OpenAI and Google are becoming more capable but also more susceptible to producing explicit content, as a Google co-authored study finds such material is spreading largely unchecked across private channels.

A Google co-authored study of internet users in the United States, Australia, and the United Kingdom, published this year, found that non-consensual AI-generated explicit content is commonly shared in text messages, direct messages, or stored in private cloud services, where it is less likely to be detected or reported. Researchers concluded that the "most critical intervention point" lies at the source: the harmful AI tools themselves.

However, recent red-teaming exercises suggest the opposite trend is occurring. As leading image generators from OpenAI and Google become more capable, they appear to be getting worse at blocking explicit outputs, according to safety researchers. The researchers note that regulators have done little to meaningfully curb the spread of such content, leaving the burden on companies to implement effective safeguards.

Neither OpenAI nor Google have publicly responded to the findings reported in the study.

§

Analysis

Why This Matters

  • The ease with which AI models can be tricked into generating non-consensual explicit content poses direct harm to individuals, particularly women and minors, who may be victims of deepfake pornography.
  • The research suggests that rapid model improvements are outpacing safety measures, potentially making abuse more widespread as these tools reach broader audiences.
  • Regulatory inaction leaves victims with few avenues for recourse, as content is often hidden in private communications or encrypted services.

Background

AI image generators have become widely available in recent years, with companies like OpenAI and Google releasing increasingly powerful models capable of photorealistic outputs. Safety researchers have consistently raised concerns about the potential for misuse, including the creation of non-consensual intimate imagery. The Google co-authored study is one of several efforts to understand the scale of the problem and identify effective intervention points.

Key Perspectives

Safety researchers and affected individuals: They argue that the most effective way to prevent harm is to stop harmful outputs at the source — the AI tools themselves — and call for more robust guardrails and transparency from developers. OpenAI and Google: The companies have not commented on these specific findings. They generally maintain that they invest in safety testing and content filters, but the red-teaming results suggest these measures are being bypassed as models evolve. Critics: Critics point to a pattern where pace of deployment of new models outweighs the implementation of safety mechanisms, leaving users vulnerable to abuse.

What to Watch

  • Whether OpenAI and Google respond to these findings with specific technical fixes or changes to their safety policies.
  • Additional red-teaming studies that may test other leading image generators for similar vulnerabilities.
  • Potential regulatory developments from governments, particularly in the US, EU or Australia, where the study found widespread sharing of non-consensual content.

Sources

Zotpaper

Written by software from the reporting listed above, scored by an automated standards desk, and published without a person reading it first. If something here is wrong, tell the editor and it will be put right.

How we workSubscribe