Action taken after Centre raises concerns over AI-generated obscene content and compliance with Indian laws. The government of India has acted upon spotting mass distribution of obscene content on X, the social media platform that was previously referred to as Twitter.
According to the officials, a lot of the content was generated through the use of artificial intelligence software and was in contravention to the Indian laws.
X censored approximately 3,500 posts, as well as suspended over 600 accounts, following the flagging of explicit and objectionable content on the platform by the Centre. The company also gave an assurance to the government that they would abide by the Indian regulations and ensure that such material does not come up in the future.
This problem arisen last week when the Ministry of Electronics and Information Technology provided a formal notice to the X Corp. The ministry noted that it has witnessed the misuse of AI tools by users to create and share obscene content and videos and most of them were derogatory against women.
An AI chatbot named Grok made by xAI and integrated with X was raised by the government as users were creating explicit and non-consensual content, including deepfake-style work, using prompts and image manipulation techniques. There were also reports of fake accounts that were used to share such content.
X had 72 hours to give an action taken report. Failure to comply would mean that the platform risks losing legal protection under Section 79 of the Information Technology Act as well as legal action under the laws governing indecent representation of women and child protection.
After the notice, X deleted posts with flags and accounts that were associated with the abuse. The company also admitted the holes in the enforcement and stated that it would tighten its content moderation mechanisms.
Therefore, the ministry has ordered X to perform an extensive audit of Grok’s design and protective measures in order to confirm that AI cannot produce any kind of illegal or obscene product.
The government will also require that all users who are found to be breaking Grok’s rules, whether they upload the material directly or create and modify it using an AI tool, be subject to harsh penalties, including temporary or permanent bans for severe incidents.
Government officials stated that these rules will apply to all users. Content created by users who use AI tools is included under the rules regarding item uploaded directly by the user. Furthermore, as AI technology continues to develop, social media will be responsible for ensuring that users do not misuse this technology.
The government is urging all companies operating in India to ensure they have adequate security measures in place and are complying with the country’s law regulations to remain compliant in relation to protecting users from harmful and illegal content.
