India has instructed Elon Musk’s X to promptly implement technical and procedural modifications to its AI chatbot Grok after users and lawmakers highlighted the creation of “obscene” content, including AI-manipulated images of women produced using the tool.

On Friday, India’s IT ministry issued the directive compelling Musk’s X to take corrective measures on Grok, including limiting the generation of content that involves “nudity, sexualization, sexually explicit, or otherwise unlawful” material. The ministry also imposed a 72-hour deadline for the social media platform to provide an action-taken report outlining the measures it has adopted to avert the hosting or distribution of content classified as “obscene, pornographic, vulgar, indecent, sexually explicit, pedophilic, or otherwise banned under the law.”

The directive, examined by TechCrunch, cautioned that noncompliance could undermine X’s “safe harbor” protections — legal safeguards against liability for user-generated content under Indian legislation.

India’s action follows concerns expressed by users who presented examples of Grok being instructed to modify images of individuals — mainly women — to give the impression that they were in bikinis, leading to a formal grievance from Indian parliamentarian Priyanka Chaturvedi. Additionally, recent reports highlighted cases where the AI chatbot produced sexualized images involving minors, a matter that X recognized earlier on Friday as a result of failures in their safety measures. Those images were subsequently removed.

However, images produced using Grok that made women seem to be in bikinis through AI modification remained publicly accessible on X at the time of publication, as noted by TechCrunch.

This latest order follows the Indian IT ministry’s issuance of a wider advisory on Monday, also reviewed by TechCrunch, to social media platforms, reminding them that adherence to local regulations regarding obscene and sexually explicit content is essential for maintaining legal immunity from liability for user-generated material. The advisory encouraged companies to bolster their internal safeguards and warned that failure to comply could result in legal repercussions under India’s IT and criminal laws.

“It is reiterated that non-compliance with the above requirements shall be viewed seriously and may result in strict legal consequences against your platform, its responsible officers and the users on the platform who violate the law, without any further notice,” the directive cautioned.

The Indian government stated that noncompliance could prompt action against X under India’s IT law and criminal regulations.

India, recognized as one of the largest digital markets globally, has become a crucial test case for the extent to which governments will hold platforms accountable for AI-generated content. Any increase in enforcement within the nation may have significant consequences for international technology firms operating in various jurisdictions.

The directive arises as Musk’s X continues to legally challenge certain components of India’s content regulation laws, contending that federal government takedown powers risk encroachment, even while the platform has complied with most blocking orders. Concurrently, Grok has seen increasing usage by X users for instant fact-checking and commentary on news stories, rendering its outputs more visible — and politically charged — than those of standalone AI systems.

X and xAI did not respond promptly to inquiries regarding the Indian government’s directive.