The social media network X has provoked major outrage after people utilized its artificial intelligence assistant Grok to alter pictures of females by removing their clothing.
Numerous examples have surfaced where the technology has been employed to remove clothing from women to make them appear in swimwear non-consensually, as well as to place them in inappropriate scenarios.
Company Response and Legal Position
XAI, the firm behind Grok, offered no comment to inquiries, aside from an automatically-generated message stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to outlaw image stripping technology. Under a proposed criminal offence, anyone who provided such technology would "receive a jail term and substantial penalties".
"X or Grok is capable of halting these types of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said platforms must "assess the danger" of people seeing unlawful material on their networks, but stopped short of confirming whether it was currently examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some premium features—which reacts to X users' queries when they reference it in a post.
It is frequently used to offer commentary or additional background to other posters' remarks, but people on X are also can alter an posted image through its AI image editing function.
History of Criticism and Policy
The tool has been criticised for permitting users to produce images and footage with nudity and sexually suggestive content, and it was previously accused of making graphic content of celebrities.
"The platform has been allowing the generation and distribution of these images for a long time without taking any steps and we have not witnessed any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "showing images of persons in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or share non-consensual intimate images or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said platforms such as X were required to take "suitable steps" to "lessen the risk" of users coming across illegal content on their platforms and to delete it quickly when they become aware of it.
