Discord acknowledges that an AI moderation error unjustly banned users for innocuous images

Discord acknowledges that an AI moderation error unjustly banned users for innocuous images

Discord has recognized that an issue in its AI moderation framework erroneously banned over 8,000 users in the last two months, as harmless images—including spreadsheets, chessboards, gaming textures, plus white and gray transparent backgrounds—were wrongly identified as harmful material.

The corporation confirmed that this problem had been impacting accounts since May, with an extra 200 users banned over the weekend before the team identified and resolved the issue. All impacted accounts are in the process of being reinstated.

This situation underscores one of the escalating challenges related to AI-driven moderation as many platforms increasingly depend on automated systems to detect illegal or abusive content at a large scale.

In an extensive thread on X, Discord clarified that its automated safety system functions by comparing uploaded content against known harmful material databases. Although the technology aims to identify illegal content, the company admitted that false positives can sometimes occur. A human moderator assesses the content, but a glitch led to the immediate banning of affected accounts.

“We’re working on improved safeguards to ensure this doesn’t happen again,” the company stated. 

On X and Reddit, users reported being permanently banned merely for uploading images with square grid patterns. Many users theorized that Discord’s AI moderation tools have grown increasingly sensitive to grid-like designs because they have been used before to camouflage or conceal NSFW and child exploitation material from automated detection systems. 

Impacted users have expressed their anger on social media, with some claiming that permanent account bans based exclusively on automated detection can lead to serious repercussions, especially for individuals who depend on Discord for work, gaming communities, or long-distance social interactions.

“Losing a Discord account over something as unjust as this can be highly damaging and significantly impact users, and daily millions are affected by erroneous AI bans. This must be stopped,” a user on X commented. 

Discord is not the only platform facing moderation issues due to automated systems. Last year, Instagram and Facebook Groups users reported numerous unexplained account suspensions that many suspected were due to AI moderation mechanisms. Although users pointed to automation as the probable cause, Meta never publicly confirmed if AI errors were to blame. Now, Meta’s Oversight Board is advocating for greater transparency.

Tumblr also encountered complaints from users last year who claimed their accounts were mass-suspended without clear justifications.

When you make purchases through links in our articles, we may receive a small commission. This doesn’t influence our editorial independence.