The social media network X has sparked major backlash after individuals used its AI chatbot Grok to change pictures of females by stripping their attire.
Several instances have been seen where the tool has been employed to virtually strip female subjects to show them in swimwear without permission, as well as to place them in inappropriate contexts.
Company Reaction and Regulatory Stance
XAI, the organization responsible for Grok, offered no comment to requests for comment, except for an automatically-generated message stating "legacy media lies".
A government official stated that new laws is being brought forward to outlaw image stripping software. Under a upcoming law, anyone who distributed such tools would "be subject to a prison sentence and substantial fines".
"X or Grok has the ability to stop these forms of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Position and Tool Functionality
The communications watchdog Ofcom said technology companies must "assess the risk" of people seeing unlawful material on their platforms, but stopped short of confirming whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a free AI assistant—with some premium functions—which answers X users' requests when they mention it in a post.
It is often used to provide commentary or more background to other users' comments, but people on X are also able to alter an uploaded image through its AI image editing function.
Background of Controversy and Policy
The chatbot has been faulted for permitting users to generate photos and videos with nudity and sexualised material, and it was earlier accused of making sexually explicit material of public figures.
"The platform has been allowing the creation and spread of these pictures for a long time without taking any action and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own terms of service forbids "portraying likenesses of individuals in a pornographic manner".
In a statement, Ofcom said it was illegal to "make or share private pictures or child sexual abuse material" and stated this covered sexual deepfakes created with AI.
It said services such as X were required to take "suitable steps" to "reduce the danger" of users viewing unlawful content on their services and to delete it quickly when they become aware of it.
