California Attorney General Orders xAI to Halt Production of Nonconsensual Sexual Deepfakes
In a decisive move against the proliferation of nonconsensual sexual imagery generated by artificial intelligence, California Attorney General Rob Bonta has issued a cease-and-desist order to xAI, the AI startup founded by Elon Musk. This action follows reports that xAI’s chatbot, Grok, has been utilized to create explicit deepfake images of women and minors without their consent.
Earlier this week, the Attorney General’s office announced an investigation into xAI concerning these allegations. On Friday, the office escalated its response by demanding that xAI immediately cease the creation and distribution of nonconsensual intimate images and child sexual abuse material (CSAM).
Today, I sent xAI a cease-and-desist letter, demanding the company immediately stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material, stated Attorney General Bonta. The creation of this material is illegal. I fully expect xAI to immediately comply. California has zero tolerance for [CSAM].
The Attorney General’s office further alleged that xAI appears to be facilitating the large-scale production of nonconsensual nude images, which are being used to harass women and girls across the internet. The agency has given xAI a five-day deadline to demonstrate the steps it is taking to address these issues.
Central to the controversy is Grok’s spicy mode feature, designed to generate explicit content. This feature has drawn international scrutiny, with countries such as Japan, Canada, and Britain initiating investigations into Grok’s activities. Additionally, Malaysia and Indonesia have temporarily blocked access to the platform. Despite xAI implementing some restrictions on its image-editing features earlier this week, the California Attorney General’s office proceeded with the cease-and-desist order.
X’s safety account has previously condemned such user activities, stating: Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content. When TechCrunch reached out to xAI for comment, the company responded with an automated email stating Legacy Media Lies. Further inquiries to the California Attorney General’s office are ongoing.
The rise of free generative AI tools has led to a disturbing increase in nonconsensual sexual material. This issue extends beyond xAI, affecting multiple platforms and drawing attention from both state leaders and Congress. Recently, lawmakers sent a letter to executives of several companies—including X, Reddit, Snap, TikTok, Alphabet, and Meta—inquiring about their plans to combat the spread of sexualized deepfakes.
