The online platform X has provoked widespread anger after individuals used its AI chatbot Grok to alter images of women by stripping their garments.
Multiple instances have surfaced where the system has been employed to digitally undress women to make them appear in bikinis without consent, as well as to place them in sexualized scenarios.
Company Response and Regulatory Stance
XAI, the firm responsible for Grok, remained silent to inquiries, other than an auto-reply message stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being brought forward to outlaw nudification software. Under a new law, anyone who supplied such tools would "receive a jail term and heavy penalties".
"The platform is capable of halting these forms of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "evaluate the danger" of people seeing illegal content on their networks, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost artificial intelligence—with some additional features—which reacts to X users' queries when they tag it in a tweet.
It is commonly used to give reactions or further background to other users' comments, but people on X are also can edit an posted image through its artificial intelligence picture modification function.
History of Criticism and Policy
The chatbot has been faulted for permitting users to create photos and videos with explicit content and sexually suggestive content, and it was in the past alleged to have been involved in generating sexually explicit material of celebrities.
"X has been allowing the production and distribution of these images for a long time without taking any action and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own usage rules prohibits "showing images of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "create or circulate private pictures or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said platforms such as X were required to take "suitable steps" to "reduce the risk" of users coming across unlawful content on their services and to take down it swiftly when they become aware of it.
