Apple tightens App Store rules to block AI 'nudify' bait-and-switch apps

Policy change follows San Francisco order and watchdog report on AI-generated nude apps

By LineZotpaper
Published
Read Time2 min
Apple has updated its App Store developer guidelines to ban apps that use AI to create offensive content and to block so-called nudify apps that hide their real purpose during the review process. The company said it will remove apps that reveal harmful capabilities, such as nudification, only after they have been approved.

Apple tightened its App Store developer guidelines over the weekend to ban apps that use AI to create offensive content, as well as apps that masquerade as ordinary programs during review and later turn into nudify apps or surface those features prominently.

The updated rules require apps that host user-generated content to include preventative measures to moderate objectionable material. "Apps that host user-generated content are required to filter objectionable material, give users a way to report offensive content, and let users block abusive accounts," the company said in a blog post.

Apple also said apps containing hidden or undocumented features are prohibited, including apps that misrepresent their purpose during review and only reveal harmful capabilities, such as "nudification," after approval, a tactic it described as bait-and-switch. The company said it will remove apps that add features not present in the version submitted for App Store review, and it has reinforced its legal language to make clear it will bar apps promoting human trafficking and the exploitation of children.

Apple said its review process blocked more than 150,000 apps from reaching users last year and removed 193,000 developer accounts for fraud and abuse.

The changes follow growing pressure over AI-based nudification apps. In July, San Francisco City Attorney David Chiu ordered Apple and Google to clean up and delete such apps from their stores, accusing the companies of profiting from them through store fees. In April, a Tech Transparency Project report found that both stores hosted multiple nudify apps, some of which were rated as suitable for minors.

§

Analysis

Why This Matters

  • The App Store is the only sanctioned way to install apps on iPhones, so Apple's guidelines directly shape what millions of users can access.
  • The policy targets an app category that produces realistic nude images without the subject's consent, and the new rules may set a precedent for how stores handle AI-generated harmful content.
  • Enforcement will determine whether the updated language changes what actually appears in the store.

Background

Nudify apps use AI to edit photos of people into nude images, often without their knowledge or consent. The category has drawn increasing scrutiny from regulators and researchers as AI image tools have become more widely available. Apple's announcement is the latest in a series of actions, following San Francisco's July order to both Apple and Google and the April report from the Tech Transparency Project that documented such apps on both stores. Apple says its review process already blocks large numbers of problematic apps, and it has now updated its written rules to explicitly cover misleading apps and AI-generated offensive content.

Key Perspectives

Apple: The company says its guidelines require moderation controls for user-generated content and prohibit hidden features, and it will remove apps that add functions not included in the version that was reviewed. It frames the update as a continuation of its existing safety review work.

San Francisco City Attorney David Chiu: Chiu ordered Apple and Google to remove nudification apps, accusing the companies of profiting from the apps through App Store fees.

Tech Transparency Project: The watchdog group's report found nudify apps on both Apple's and Google's stores, including some rated as suitable for minors, raising questions about how effectively the stores screened the apps.

What to Watch

  • Whether Apple actively removes existing nudify apps and how consistently it applies the new rules to new submissions.
  • Whether Google introduces similar changes to Google Play, given it received the same order from San Francisco.
  • Whether regulators or lawmakers pursue further action against app stores if nudify apps remain available.

Sources

Checked

11 claims in this story checked against 1 source: 10 supported.

Show each claim
  1. SupportedApple tightened App Store rules to block AI 'nudify' bait-and-switch apps.TechCrunch: “Apple tightened its developer guidelines over the weekend to ban apps that use AI to create offensive content, as well as those that masquerade as regular apps to pass the App Store review process and then turn into nudify apps, or surface those features prominently.”
  2. SupportedPolicy change follows San Francisco order and watchdog report on AI-generated nude apps.TechCrunch: “In July, San Francisco ordered both Apple and Google to clean up and delete nudification apps from their app stores. City Attorney David Chiu accused the companies of profiting from these apps through store fees. And in April, a Tech Transparency Project report indicated that both tech giants’ stores hosted multiple nudify apps, some of which were rated as suitable for minors.”
  3. SupportedApple updated guidelines to ban apps that use AI to create offensive content and to block nudify apps that hide their real purpose during review.TechCrunch: “Apple tightened its developer guidelines over the weekend to ban apps that use AI to create offensive content, as well as those that masquerade as regular apps to pass the App Store review process and then turn into nudify apps”
  4. SupportedApple said it will remove apps that reveal harmful capabilities, such as nudification, only after they have been approved.TechCrunch: “Apple said that it will remove apps that add features not present in the version submitted for App Store review. ... apps that misrepresent their purpose during review and only reveal harmful capabilities — such as 'nudification' — after approval”
  5. Supported"Apps that host user-generated content are required to filter objectionable material, give users a way to report offensive content, and let users block abusive accounts," the company said in a blog post.TechCrunch: ““Apps that host user-generated content are required to filter objectionable material, give users a way to report offensive content, and let users block abusive accounts."”
  6. SupportedApple said apps containing hidden or undocumented features are prohibited, including apps that misrepresent their purpose during review and only reveal harmful capabilities, such as 'nudification,' after approval, a tactic it described as bait-and-switch.TechCrunch: “Apps containing hidden or undocumented features are also prohibited on the App Store, including apps that misrepresent their purpose during review and only reveal harmful capabilities — such as 'nudification' — after approval, a tactic commonly known as bait-and-switch”
  7. SupportedApple said it will remove apps that add features not present in the version submitted for App Store review, and it has reinforced its legal language to make clear it will bar apps promoting human trafficking and the exploitation of children.TechCrunch: “Apple said that it will remove apps that add features not present in the version submitted for App Store review. The iPhone maker has also updated the language in its legal section to reinforce that it will bar apps promoting human trafficking and exploitation of children.”
  8. SupportedApple said its review process blocked more than 150,000 apps from reaching users last year and removed 193,000 developer accounts for fraud and abuse.TechCrunch: “Apple said last year its review process blocked over 150,000 apps from reaching users, and removed 193,000 developer accounts on account of fraud and abuse.”
  9. Partly supportedThe changes follow growing pressure over AI-based nudification apps.TechCrunch: “In July, San Francisco ordered... And in April, a Tech Transparency Project report indicated...”
  10. SupportedIn July, San Francisco City Attorney David Chiu ordered Apple and Google to clean up and delete such apps from their stores, accusing the companies of profiting from them through store fees.TechCrunch: “In July, San Francisco ordered both Apple and Google to clean up and delete nudification apps from their app stores. City Attorney David Chiu accused the companies of profiting from these apps through store fees.”
  11. SupportedIn April, a Tech Transparency Project report found that both stores hosted multiple nudify apps, some of which were rated as suitable for minors.TechCrunch: “And in April, a Tech Transparency Project report indicated that both tech giants’ stores hosted multiple nudify apps, some of which were rated as suitable for minors.”

Zotpaper

Written by Rex Doyle on the Tech & AI desk. Claims checked by Ines Fairley, verification. Read for legal risk by Marcus Hale, legal. Edited by Angus Burke, chief sub.

1 source10 of 11 claims matched to a sourcefirst published 19 Oct, 11:53 pm
Meet the newsroomHow we workSubscribe