The online network X has caused significant backlash after users used its AI chatbot Grok to alter images of women by removing their garments.
Several instances have emerged where the tool has been employed to virtually strip women to render them in bikinis without consent, as well as to put them in sexually explicit scenarios.
Developer Response and Legal Stance
XAI, the company that created Grok, offered no comment to inquiries, other than an standard message stating "legacy media lies".
A Home Office representative stated that legislation is being proposed to prohibit nudification software. Under a new law, anyone who distributed such tools would "receive a prison sentence and significant penalties".
"X or Grok is capable of halting these forms of misuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Functionality
The regulator Ofcom said technology companies must "assess the risk" of people accessing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in regarding artificial intelligence images.
Grok is a no-cost chatbot—with some additional functions—which answers X users' prompts when they mention it in a message.
It is commonly used to provide reactions or further background to other users' remarks, but people on X are also capable of modify an posted photo through its artificial intelligence picture modification tool.
History of Criticism and Rules
The chatbot has been condemned for allowing users to generate photos and footage with nudity and sexualised content, and it was earlier accused of generating inappropriate material of public figures.
"X has been permitting the generation and distribution of these images for months without taking any action and we have not seen any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy forbids "depicting likenesses of individuals in a graphic manner".
In a statement, Ofcom said it was illegal to "make or share deepfakes or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said networks such as X were mandated to take "appropriate actions" to "reduce the danger" of users encountering unlawful content on their services and to take down it swiftly when they learn of it.
