Apple has recently removed the AI-powered application Kromix from its App Store. This decision came after the app was found to be promoting explicit deepfake content through advertisements on Meta’s platforms. The ads featured AI-generated pornographic images of individuals closely resembling real people, including a prominent U.S. politician.
Kromix marketed itself as an ‘AI image styler’ with ‘no restrictions,’ allowing users to create explicit deepfake images. One particular advertisement depicted a woman resembling a well-known female U.S. politician, transitioning from a formal setting to an explicit scene. This campaign targeted male users exclusively and ran 32 ads before being taken down.
Upon being alerted to the issue, Apple promptly removed Kromix from the App Store. The company stated that the app had added prohibited features after its initial approval. Apple emphasized its strict prohibition against apps designed to generate, distribute, or consume pornography, including ‘nudification’ apps, and reaffirmed its commitment to identifying and removing such content.
Meta also responded by removing the offending advertisements. This incident underscores the challenges tech companies face in monitoring and controlling the content promoted through their platforms, especially with the rapid advancement of AI technologies.
In a related development, Apple had previously removed eight similar ‘nudify’ apps from the App Store following a demand from San Francisco City Attorney David Chiu. This pattern highlights the ongoing struggle to regulate applications that misuse AI for creating non-consensual explicit content.
The removal of Kromix and similar apps raises important questions about the ethical use of AI in content creation. As AI technology becomes more sophisticated, the potential for misuse increases, necessitating stricter oversight and regulation. Both app developers and platform operators must collaborate to establish and enforce guidelines that prevent the exploitation of AI for creating harmful content.
