The online platform X has caused significant outrage after users used its artificial intelligence assistant Grok to change images of females by stripping their garments.
Numerous examples have emerged where the technology has been used to digitally undress women to make them appear in bikinis non-consensually, as well as to place them in sexually explicit situations.
Company Reaction and Regulatory Stance
XAI, the firm behind Grok, remained silent to requests for comment, except for an automatically-generated response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being introduced to ban image stripping tools. Under a upcoming criminal offence, anyone who distributed such technology would "receive a jail term and substantial fines".
"The platform is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said technology companies must "evaluate the risk" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some premium features—which answers X users' prompts when they mention it in a tweet.
It is frequently used to offer reactions or additional context to other users' remarks, but people on X are also able to edit an posted picture through its AI image editing function.
History of Criticism and Rules
The chatbot has been criticised for permitting users to produce photos and videos with explicit content and sexually suggestive material, and it was in the past accused of making inappropriate material of celebrities.
"The platform has been permitting the generation and distribution of these images for a long time without taking any action and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of people in a pornographic way".
In a statement, Ofcom said it was against the law to "make or circulate private pictures or child sexual abuse material" and stated this covered explicit AI images made with AI.
It said services such as X were obligated to take "appropriate steps" to "reduce the danger" of users viewing illegal content on their platforms and to delete it promptly when they are notified of it.
