The social media network X has caused major anger after people utilized its AI chatbot Grok to alter images of women by taking off their garments.
Several instances have been seen where the tool has been employed to digitally undress female subjects to make them appear in swimwear without consent, as well as to place them in inappropriate situations.
Developer Reaction and Regulatory Position
XAI, the firm responsible for Grok, did not respond to inquiries, except for an automatically-generated message stating "legacy media lies".
A Home Office official stated that new laws is being introduced to prohibit image stripping tools. Under a proposed law, anyone who distributed such technology would "receive a jail term and substantial fines".
"The platform has the ability to stop these forms of misuse if they chose to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said technology companies must "consider the risk" of people seeing illegal content on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in relation to artificial intelligence pictures.
Grok is a free AI assistant—with some paid-for functions—which reacts to X users' queries when they mention it in a post.
It is often used to provide reactions or additional context to other posters' remarks, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification tool.
History of Criticism and Policy
The system has been condemned for permitting users to produce photos and footage with explicit content and sexualised material, and it was previously accused of generating graphic material of celebrities.
"The platform has been permitting the creation and spread of these images for months without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "showing images of people in a graphic manner".
In a comment, Ofcom said it was against the law to "create or share deepfakes or illegal content" and stated this covered sexual deepfakes made with AI.
It said services such as X were mandated to take "necessary actions" to "reduce the danger" of users viewing illegal material on their platforms and to remove it promptly when they learn of it.
