The online network X has provoked significant anger after people employed its AI chatbot Grok to change pictures of women by stripping their clothing.
Multiple cases have surfaced where the technology has been employed to digitally undress female subjects to make them appear in bikinis without permission, as well as to place them in sexualized scenarios.
Developer Response and Regulatory Position
XAI, the company that created Grok, offered no comment to requests for comment, except for an standard message stating "legacy media lies".
A government official stated that new laws is being proposed to prohibit nudification software. Under a proposed law, anyone who distributed such technology would "be subject to a prison sentence and significant fines".
"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the danger" of people seeing illegal content on their platforms, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some paid-for features—which answers X users' requests when they tag it in a message.
It is often used to offer reactions or more context to other users' remarks, but people on X are also capable of modify an uploaded picture through its AI image editing tool.
Background of Criticism and Policy
The chatbot has been criticised for permitting users to generate images and clips with explicit content and sexualised material, and it was previously alleged to have been involved in creating sexually explicit material of public figures.
"X has been enabling the generation and sharing of these pictures for months without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "showing likenesses of people in a pornographic fashion".
In a comment, Ofcom said it was illegal to "make or share private pictures or illegal content" and confirmed this applied to explicit AI images created with AI.
It said services such as X were mandated to take "appropriate measures" to "lessen the danger" of users coming across unlawful content on their platforms and to delete it swiftly when they become aware of it.
