Roblox’s AI listens in, but humans still rule Voice Chat moderation

Online gaming is great, but let’s be honest: voice chat can sometimes devolve into a dumpster fire of insults and fury. It’s much easier to yell nasty words than to type them out, which has caused a major moderation problem. Roblox, like other major gaming platforms, is addressing this toxicity square on, and they are doing so with a potent tool: artificial intelligence. However, even with cutting-edge technology, it appears that good old human judgment still plays an important role.

Roblox’s Senior Technical Director Kiran Bhat and Voice Safety Lead Hannes Heikinheimo revealed their voice chat moderation system at the Game Developers Conference (GDC) recently. They have been utilizing machine learning for a year, and the results are quite spectacular. However, the devs still seem to prefer human moderation for an added layer of security that the Robloxians need on the platform.

Roblox devs reveal why AI-based moderation isn’t enough for voice chats filled with “toxicity”

In a talk at GDC 2025, Roblox devs talked about their experience with AI moderation on the platform. As Bhat put it, “Moderating voice in real time sounds really daunting, because to get the voice moderated, you not only have to know what the person is saying, but you also want to capture the tone and the intensity of what is being said to decide whether something is toxic or not. There’s also context that is part of whether something is offensive. So it’s a really hard problem.”

Fortunately, a surprising proportion of hazardous behavior follows consistent patterns. Bhat explained that at least 85% of toxicity fits into four primary categories and that a “vast majority” of infractions may be handled by a list of only 50 keywords. Catch those and you’ll have made tremendous progress. And that is where Machine Learning shines.

Heikinheimo emphasized the “consistency” of AI. Machines, unlike human moderators, do not experience mood swings, bias, or bad days. They merely obey the rules. This makes them especially helpful in “very clear cases” of hazardous behavior. Heikinheimo said that after a year of utilizing the method, the number of abuse reports per “active hour of talk” was cut in half. That’s a significant step toward improving the online environment.

But here’s the catch: AI isn’t flawless. While machines are great at pattern matching, they struggle with nuance, intent, and those strange edge cases that only humans seem to understand. “Humans are still better in those types of things where you might be very close to the decision boundary,” Heikinheimo made clear. “Or it might be a case that’s very rare where we don’t have that much data for the machine learning system to learn from. So in those cases, humans can still be better.”

Roblox is not alone in this regard. Other studios, such as NetEase Games’ Marvel Rivals and Activision’s Call of Duty, are also looking at AI-powered voice moderation. This appears to be becoming a trend, albeit a potentially valuable one. It makes you think: that AI is a terrific tool, but it is not a magical answer that will replace human oversight. It’s more like a powerful assistant that can handle mundane tasks, freeing up human moderators to focus on the difficult, nuanced circumstances that necessitate genuine understanding and discernment. So, while the robots are listening in, it’s encouraging to know that the humans of Roblox still play an important role in making online spaces like that less toxic.