The online network X has sparked major backlash after users employed its AI chatbot Grok to manipulate photos of women by removing their clothing.
Numerous examples have surfaced where the tool has been employed to digitally undress women to make them appear in bikinis without consent, as well as to insert them in sexualized scenarios.
Developer Response and Regulatory Stance
XAI, the organization responsible for Grok, remained silent to inquiries, aside from an automatically-generated message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to ban nudification software. Under a upcoming law, anyone who distributed such tools would "receive a jail term and significant penalties".
"X or Grok has the ability to stop these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the risk" of people accessing illegal content on their networks, but did not confirm whether it was presently investigating X or Grok in relation to AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' prompts when they reference it in a post.
It is often used to offer reactions or more context to other posters' comments, but people on X are also capable of alter an uploaded picture through its AI image editing function.
Background of Controversy and Rules
The tool has been faulted for enabling users to generate pictures and videos with nudity and sexually suggestive material, and it was earlier alleged to have been involved in making inappropriate material of celebrities.
"X has been permitting the creation and spread of these pictures for a long time without taking any action and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying images of persons in a graphic fashion".
In a statement, Ofcom said it was against the law to "make or circulate private pictures or illegal content" and clarified this included sexual deepfakes generated with AI.
It said platforms such as X were obligated to take "suitable steps" to "lessen the risk" of users encountering illegal content on their platforms and to remove it swiftly when they learn of it.
