The online platform X has provoked widespread anger after people used its AI chatbot Grok to manipulate photos of females by stripping their clothing.
Multiple examples have been seen where the system has been used to digitally undress women to render them in swimwear without consent, as well as to place them in inappropriate contexts.
Company Response and Legal Stance
XAI, the organization that created Grok, offered no comment to inquiries, aside from an automatically-generated message stating "legacy media lies".
A government official stated that legislation is being proposed to prohibit image stripping tools. Under a upcoming criminal offence, anyone who distributed such tools would "face a prison sentence and substantial fines".
"X or Grok is capable of halting these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said tech firms must "consider the risk" of people seeing unlawful material on their platforms, but did not specify whether it was currently looking into X or Grok in regarding AI-generated images.
Grok is a no-cost chatbot—with some additional functions—which answers X users' requests when they mention it in a post.
It is often used to give commentary or more background to other users' remarks, but people on X are also capable of modify an uploaded photo through its artificial intelligence picture modification function.
Background of Controversy and Rules
The tool has been condemned for permitting users to produce photos and footage with explicit content and sexually suggestive content, and it was previously alleged to have been involved in generating graphic content of celebrities.
"X has been allowing the creation and spread of these images for months without taking any steps and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own usage rules bans "portraying likenesses of people in a graphic manner".
In a statement, Ofcom said it was unlawful to "make or share private pictures or illegal content" and confirmed this included explicit AI images generated with AI.
It said networks such as X were obligated to take "suitable steps" to "minimize the danger" of users coming across unlawful material on their platforms and to delete it swiftly when they become aware of it.
