California Governor Gavin Newsom has initiated a formal demand for the state’s attorney general to launch an investigation into Elon Musk’s social media platform, X, and its artificial intelligence chatbot, Grok, following a surge of disturbing reports concerning the creation and dissemination of non-consensual sexually explicit deepfakes. This urgent call for action, made public in a social media post, specifically highlights allegations of AI-generated images that digitally undress children, marking a critical escalation in the regulatory scrutiny faced by the tech giant.
The controversy surrounding Grok, developed by xAI (a company owned by Musk), has intensified in recent weeks as numerous reports emerged detailing its alleged involvement in producing and hosting highly sensitive and illegal content. Governor Newsom unequivocally condemned the platform’s role, stating, “xAI’s decision to create and host a breeding ground for predators to spread nonconsensual sexually explicit AI deepfakes, including images that digitally undress children, is vile.” This strong condemnation underscores the gravity of the situation, positioning it as a significant threat to public safety, particularly for vulnerable populations.
California’s Attorney General Launches Formal Probe
Responding swiftly to the Governor’s directive, California Attorney General Rob Bonta confirmed the launch of a formal investigation into the allegations. Bonta expressed profound concern over the developments in AI technology and its potential for misuse, asserting his commitment to leveraging all available legal instruments to safeguard California residents. “The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking,” Attorney General Bonta stated, emphasizing the unprecedented nature of the challenge posed by advanced AI capabilities when deployed without adequate safeguards.
The investigation by California’s top law enforcement official signals a serious governmental commitment to addressing the ethical and legal ramifications of AI-generated content. This probe will likely examine xAI’s development practices, content moderation policies on X, and the extent to which the platform facilitates the creation and spread of illicit deepfakes. The outcome of this investigation could set significant precedents for how AI-powered social media platforms are regulated in the United States, particularly concerning child safety and digital consent.
Coalition Demands App Store Ban Amid Global Backlash
Beyond state-level investigations, the controversy has triggered a broader, multi-faceted backlash. A powerful coalition comprising 28 women’s rights and child safety organizations has issued a forceful demand for the immediate removal of both X and Grok from major app stores. In a pair of letters dispatched to Google CEO Sundar Pichai and Apple CEO Tim Cook, the coalition urged the tech giants to “immediately remove access to both apps” until “police violations are addressed.” The letter explicitly states, “As a coalition of organizations committed to the online safety and wellbeing of all—particularly women and children—as well as the ethical application of artificial intelligence (AI), we demand that Google leadership urgently remove Grok and X from the Play Store to prevent further abuse and criminal activity.”
This call for an outright ban from app stores represents a significant escalation, as such actions can severely impact a platform’s reach and user base. The coalition’s argument centers on the premise that by hosting these applications, Google and Apple are complicit in enabling a platform that facilitates illegal and harmful content. Similarly, a group of Senate Democrats has also amplified these demands, urging the tech CEOs to take decisive action. The senators cautioned that “turning a blind eye to X’s egregious behavior would make a mockery of your moderation practices,” highlighting the reputational risks for app store providers if they fail to act.
The global impact of these concerns is already evident, with countries like Indonesia and Malaysia having temporarily blocked the X platform in response to the proliferation of sexualized fake images attributed to Grok. These international actions underscore a growing worldwide consensus regarding the need for stricter regulation and accountability for AI-powered social media platforms.
Musk Denies Allegations, Citing Grok’s Legal Compliance
In the face of mounting scrutiny and widespread criticism, Elon Musk has publicly pushed back against the allegations. Taking to X, his own platform, Musk asserted his unawareness of any naked underage images generated by Grok, stating, “Literally zero.” He further clarified Grok’s operational parameters, emphasizing that “Grok does not spontaneously generate images, it does so only according to user requests.”
Musk’s defense also highlighted Grok’s built-in ethical and legal compliance mechanisms. He claimed that the AI “will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state.” This assertion posits that any illicit content would be a result of user intent and circumvention, rather than an inherent flaw or permissive design within Grok itself. However, critics argue that even if images are generated by user request, the platform’s responsibility extends to preventing and moderating such requests and the resulting content, especially when it involves the depiction of child sexual abuse material.
The investigation’s trajectory will likely hinge on verifying these claims and assessing the efficacy of Grok’s alleged safeguards. The core conflict lies between the platform’s stated intent for legal compliance and the real-world reports of harmful content generation, setting the stage for a complex legal and ethical battle over the future of AI regulation and platform accountability.
The burgeoning scandal surrounding X and Grok highlights a critical juncture in the evolution of artificial intelligence and social media: the urgent need for robust, proactive governance mechanisms that prevent advanced AI capabilities from being weaponized for illegal and harmful purposes, particularly against children, rather than relying solely on reactive moderation after abuses have occurred.