OpenAI Details Russian Influence Campaign Using ChatGPT to Promote Fake Think Tank

OpenAI disclosed on Tuesday that it disrupted a covert influence operation originating from Russia, where operators used ChatGPT to create content for a fictitious organization called the 'International Burke Institute.' The campaign promoted a bogus 'sovereignty' index that praised Russia and criticized Western governments, particularly the European Union and Germany.

According to OpenAI, the accounts accessed ChatGPT through VPNs routed from Russia. The generated content was disseminated primarily on Telegram in German, pushing narratives critical of the EU and the German government while presenting Russia in a favorable light. The think tank itself is entirely fabricated, with no real-world presence.

While the campaign's reach was limited — OpenAI noted it had little organic engagement — the company said the underlying infrastructure could have been scaled up over time, posing a greater threat if left unchecked. The banned accounts were part of a cluster that OpenAI identified through its threat intelligence systems.

This is the latest in a series of influence operations leveraging generative AI. Similar campaigns have been observed using large language models to produce text, images, and even fake personas at scale. OpenAI has previously banned accounts linked to state-sponsored disinformation efforts from Russia, China, and Iran.

The timing of the disclosure comes as governments worldwide grapple with the use of AI in information warfare. The European Union's Digital Services Act and emerging AI regulations, such as the EU AI Act, aim to force platforms to monitor and mitigate such misuse. However, detection remains a cat-and-mouse game: as AI-generated content becomes harder to distinguish from human-written material, influence campaigns can become more sophisticated and harder to trace.

Critics note that the impact of this particular operation was minimal — the fake think tank does not appear to have gained traction in mainstream discourse. Yet the pattern is concerning: AI lowers the cost of producing coherent, multilingual propaganda, enabling state actors to run multiple small-scale campaigns simultaneously, avoiding detection until after the fact.

OpenAI stated it is sharing its findings with industry partners and law enforcement agencies. The company continues to invest in detection systems that identify coordinated inauthentic behavior rather than relying solely on content-based signals.

Why This Matters

Democratization of disinformation: AI tools like ChatGPT drastically lower the cost and effort needed to produce coherent propaganda in multiple languages, enabling even small state-backed teams to run influence campaigns.
Scalability risk: Even a campaign with low initial reach can be scaled up quickly. The underlying infrastructure — VPN access, content templates, distribution channels — can be reused, making disruption crucial early on.
Regulatory implications: This case highlights the challenges of enforcing emerging AI regulations (e.g., EU AI Act) that require providers to assess and mitigate risks of misuse, including influence operations.

Background

OpenAI has been tracking state-aligned influence operations for years, publishing periodic threat intelligence reports. In 2024, the company banned accounts linked to Iranian and Chinese disinformation campaigns. This latest operation is notable for its use of a fabricated think tank — the 'International Burke Institute' — and a 'sovereignty' index designed to lend faux academic credibility.

The campaign targeted German-speaking audiences, a demographic of strategic interest to Russia given Germany's role in EU sanctions and its support for Ukraine. The use of Telegram is consistent with previous Russian influence efforts, as the platform allows for loosely moderated group chats and channel broadcasting.

Key Perspectives

OpenAI and security researchers: The campaign was detected early due to behavioral analysis and infrastructure tracking, not because of the content itself. OpenAI argues that proactive disruption is necessary even when reach is low, to deny adversaries a foothold.

Policy makers and regulators: The case underscores the need for transparency and mandatory reporting of AI misuse. Some argue platforms should not only ban accounts but also publicly attribute campaigns to hold state sponsors accountable.

Skeptics: Critics point out that the campaign had negligible impact — the fake think tank gained little visibility. They caution against overreaction that could lead to excessive content moderation or censorship. Some also note that attribution of state sponsorship can be murky and may be politicized.

What to Watch

Future OpenAI threat reports: Whether the company expands detection to include other generative AI features like images or code that could be used in influence operations.
EU AI Act enforcement: Whether this incident triggers any formal investigation or regulatory response from European authorities.
Copycat campaigns: Similar low-cost operations using other AI models (e.g., Claude, Gemini) could emerge as adversaries adapt to OpenAI's detection methods.

ZOTPAPER