The social media network X has sparked major anger after people utilized its AI chatbot Grok to manipulate photos of females by taking off their attire.
Numerous examples have surfaced where the system has been used to virtually strip women to make them appear in bikinis without permission, as well as to put them in sexually explicit situations.
Company Response and Legal Stance
XAI, the firm responsible for Grok, remained silent to inquiries, except for an standard response stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to ban nudification technology. Under a new criminal offence, anyone who supplied such tools would "receive a prison sentence and heavy penalties".
"X or Grok has the ability to stop these types of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said platforms must "consider the risk" of people accessing unlawful material on their platforms, but did not specify whether it was at this time looking into X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium functions—which responds to X users' queries when they tag it in a post.
It is frequently used to provide commentary or further background to other posters' comments, but people on X are also can edit an uploaded picture through its AI image editing function.
Background of Criticism and Rules
The tool has been faulted for enabling users to generate pictures and videos with explicit content and sexualised material, and it was earlier accused of creating sexually explicit content of celebrities.
"The platform has been enabling the production and spread of these images for a long time without taking any steps and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service forbids "portraying likenesses of individuals in a graphic fashion".
In a statement, Ofcom said it was illegal to "make or circulate private pictures or child sexual abuse material" and confirmed this applied to sexual deepfakes made with AI.
It said services such as X were mandated to take "necessary measures" to "lessen the danger" of users viewing illegal content on their platforms and to take down it promptly when they learn of it.
