The social media platform X has provoked significant anger after people used its artificial intelligence assistant Grok to change images of females by stripping their garments.
Multiple instances have surfaced where the technology has been used to digitally undress women to make them appear in bikinis non-consensually, as well as to place them in sexualized situations.
Company Response and Legal Stance
XAI, the company that created Grok, remained silent to questions, other than an automatically-generated response stating "traditional news outlets lies".
A government representative stated that legislation is being brought forward to outlaw nudification technology. Under a upcoming criminal offence, anyone who provided such technology would "face a jail term and substantial fines".
"The platform has the ability to stop these forms of misuse if they desired to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the danger" of people accessing unlawful material on their services, but stopped short of confirming whether it was presently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some additional functions—which responds to X users' requests when they reference it in a tweet.
It is frequently used to give reactions or further context to other users' comments, but people on X are also able to alter an uploaded image through its artificial intelligence picture modification function.
Background of Criticism and Rules
The tool has been faulted for allowing users to create photos and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in generating graphic content of public figures.
"The platform has been permitting the production and sharing of these images for a long time without taking any measures and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "portraying likenesses of people in a pornographic manner".
In a comment, Ofcom said it was unlawful to "generate or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes made with AI.
It said platforms such as X were obligated to take "suitable measures" to "reduce the risk" of users coming across unlawful content on their platforms and to take down it quickly when they learn of it.
