Elon Musk's social media network X has provoked major backlash after people used its AI chatbot Grok to manipulate images of women by stripping their garments.
Numerous instances have emerged where the tool has been employed to digitally undress women to make them appear in swimwear non-consensually, as well as to place them in sexually explicit situations.
Developer Response and Regulatory Position
XAI, the company responsible for Grok, did not respond to requests for comment, aside from an automatically-generated message stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to ban nudification tools. Under a upcoming criminal offence, anyone who distributed such tools would "be subject to a prison sentence and substantial penalties".
"X or Grok has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people seeing illegal content on their services, but did not specify whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some additional features—which responds to X users' requests when they tag it in a post.
It is commonly used to offer commentary or more context to other posters' remarks, but people on X are also capable of alter an posted photo through its AI image editing function.
Background of Criticism and Policy
The chatbot has been criticised for enabling users to create images and videos with nudity and sexualised content, and it was earlier accused of creating inappropriate content of public figures.
"X has been permitting the creation and distribution of these pictures for months without taking any measures and we have yet to see any challenge by regulators," the expert added.
XAI's own usage rules prohibits "showing images of individuals in a graphic fashion".
In a comment, Ofcom said it was against the law to "make or distribute private pictures or child sexual abuse material" and clarified this covered explicit AI images made with AI.
It said networks such as X were required to take "appropriate measures" to "minimize the danger" of users coming across unlawful material on their platforms and to take down it quickly when they learn of it.
