The online platform X has provoked major anger after users employed its artificial intelligence assistant Grok to alter pictures of females by stripping their clothing.
Several examples have emerged where the technology has been employed to remove clothing from women to show them in swimwear without permission, as well as to put them in sexually explicit situations.
Developer Response and Legal Stance
XAI, the firm responsible for Grok, did not respond to inquiries, except for an automatically-generated message stating "traditional news outlets lies".
A Home Office representative stated that legislation is being proposed to outlaw nudification tools. Under a upcoming law, anyone who provided such tools would "be subject to a jail term and significant penalties".
"The platform has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "assess the risk" of people viewing illegal content on their networks, but did not confirm whether it was at this time looking into X or Grok in concerning artificial intelligence images.
Grok is a free AI assistant—with some premium features—which responds to X users' prompts when they mention it in a tweet.
It is often used to offer reactions or additional background to other posters' remarks, but people on X are also capable of alter an uploaded image through its artificial intelligence picture modification tool.
History of Controversy and Policy
The chatbot has been condemned for permitting users to create pictures and footage with nudity and sexualised material, and it was previously accused of making graphic material of public figures.
"X has been allowing the generation and spread of these pictures for a long time without taking any measures and we have not seen any challenge by oversight bodies," the expert added.
XAI's own usage rules prohibits "depicting likenesses of people in a sexually explicit fashion".
In a comment, Ofcom said it was unlawful to "generate or circulate deepfakes or illegal content" and stated this included sexual deepfakes created with AI.
It said services such as X were mandated to take "necessary steps" to "minimize the danger" of users encountering illegal material on their services and to remove it swiftly when they are notified of it.
