{"id":3489147,"date":"2026-04-03T14:00:15","date_gmt":"2026-04-03T14:00:15","guid":{"rendered":"https:\/\/techingeek.com\/index.php\/2026\/04\/03\/the-facebook-insider-developing-content-regulation-for-the-ai-age\/"},"modified":"2026-04-03T14:00:15","modified_gmt":"2026-04-03T14:00:15","slug":"the-facebook-insider-developing-content-regulation-for-the-ai-age","status":"publish","type":"post","link":"https:\/\/techingeek.com\/index.php\/2026\/04\/03\/the-facebook-insider-developing-content-regulation-for-the-ai-age\/","title":{"rendered":"The Facebook insider developing content regulation for the AI age"},"content":{"rendered":"<div><img decoding=\"async\" src=\"https:\/\/techingeek.com\/wp-content\/uploads\/2026\/04\/the-facebook-insider-developing-content-regulation-for-the-ai-age.png\" class=\"ff-og-image-inserted\"><\/div>\n<p id=\"speakable-summary\" class=\"wp-block-paragraph\">When Brett Levenson departed Apple in 2019 to oversee business integrity at Facebook, the social media platform was amid the aftermath of the Cambridge Analytica scandal. Initially, he believed he could rectify Facebook\u2019s content moderation issues through enhanced technology.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">However, he soon discovered that the issue was more profound than just technology. Human reviewers were tasked with memorizing a 40-page policy document that had been translated into their native language by a machine, he explained. They then had approximately 30 seconds for each piece of flagged content to determine not only if it breached the rules but also what action to take: block it, ban the user, or limit its distribution. According to Levenson, those rapid decisions were only \u201cslightly better than 50% accurate.\u201d<\/p>\n<p class=\"wp-block-paragraph\">\u201cIt was almost like flipping a coin to see if the human reviewers could correctly interpret the policies, and this was after several days of the harm already occurring,\u201d Levenson stated to TechCrunch.<\/p>\n<p class=\"wp-block-paragraph\">Such a delayed, reactive method is not viable in a landscape filled with agile and well-funded adversaries. The emergence of AI chatbots has only amplified the situation, as failures in content moderation have resulted in a series of notable incidents, including chatbots directing teens toward self-harm or AI-generated images bypassing safety filters.<\/p>\n<p class=\"wp-block-paragraph\">Levenson\u2019s dissatisfaction prompted the concept of \u201cpolicy as code\u201d \u2014 a method to convert static policy documents into actionable, updatable logic closely tied to enforcement. This insight led to the establishment of Moonbounce, which has disclosed a $12 million funding raise on Friday, as learned exclusively by TechCrunch. The funding round was co-led by Amplify Partners and StepStone Group.<\/p>\n<p class=\"wp-block-paragraph\">Moonbounce collaborates with businesses to add an extra layer of safety wherever content is produced, whether by a human or by AI. The company has developed its own large language model to examine a client\u2019s policy documents, assess content in real-time, provide a response in 300 milliseconds or less, and take appropriate action. Depending on the client\u2019s preferences, this action might involve Moonbounce\u2019s system delaying distribution while the content awaits a human review or could result in blocking high-risk content immediately.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">Currently, Moonbounce caters to three primary sectors: platforms handling user-generated content such as dating applications; AI companies creating characters or companions; and AI image generation services.\u00a0<\/p>\n<div class=\"wp-block-techcrunch-inline-cta\">\n<div class=\"inline-cta__wrapper\">\n<p>Techcrunch event<\/p>\n<div class=\"inline-cta__content\">\n<p>\n\t\t\t\t\t\t\t\t\t<span class=\"inline-cta__location\">San Francisco, CA<\/span><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"inline-cta__separator\">|<\/span><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"inline-cta__date\">October 13-15, 2026<\/span>\n\t\t\t\t\t\t\t<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n<p class=\"wp-block-paragraph\">Moonbounce is facilitating over 40 million daily reviews and caters to more than 100 million daily active users on the platform, stated Levenson. Clients include AI companion startup Channel AI, image and video generation firm Civitai, and character roleplay platforms Dippy AI and Moescape.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\u201cSafety can actually serve as a product advantage,\u201d Levenson remarked to TechCrunch. \u201cIt has never been perceived this way before because it has always been an afterthought, not something that can be integrated into the product. We observe that our clients are discovering fascinating and innovative methods to leverage our technology to make safety a competitive edge and part of their product narrative.\u201d<\/p>\n<p class=\"wp-block-paragraph\">Tinder\u2019s trust and safety lead recently discussed how the dating service utilizes these types of LLM-powered solutions to achieve a tenfold improvement in detection accuracy.<\/p>\n<p class=\"wp-block-paragraph\">\u201cContent moderation has always been a challenge for major online platforms, but with LLMs now central to every application, this challenge has become even more formidable,\u201d commented Lenny Pruss, general partner at Amplify Partners, in a statement. \u201cWe chose to invest in Moonbounce because we envision a future where objective, real-time guardrails become the essential infrastructure of every AI-mediated application.\u201d<\/p>\n<p class=\"wp-block-paragraph\">AI enterprises are under increasing legal and reputational pressure after chatbots have been criticized for guiding teenagers and vulnerable users towards suicidal thoughts, with image generators like xAI\u2019s Grok being used to create non-consensual nude images. Clearly, internal safety measures are failing, and it\u2019s turning into a liability issue. Levenson mentioned that AI companies are progressively seeking assistance from external sources to enhance their safety frameworks.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\u201cWe act as a third party positioned between the user and the chatbot, which means our system isn\u2019t overwhelmed with context in the same way the chat is,\u201d Levenson explained. \u201cThe chatbot must remember, possibly, tens of thousands of tokens that have been exchanged previously\u2026Our sole concern is enforcing the rules in real-time.\u201d<\/p>\n<p class=\"wp-block-paragraph\">Levenson co-manages the 12-person company with his former Apple associate Ash Bhardwaj, who previously developed large-scale cloud and AI infrastructure across Apple\u2019s core products. Their upcoming focus is a feature known as \u201citerative steering,\u201d created in response to incidents like the 2024 suicide of a 14-year-old boy from Florida who became fixated on a Character AI chatbot. Instead of an outright refusal when harmful subjects come up, the system would intervene in the conversation and reroute it, adjusting prompts in real time to steer the chatbot toward a more actively supportive response.<\/p>\n<p class=\"wp-block-paragraph\">\u201cWe hope to incorporate into our action toolkit the capability to guide the chatbot in a more positive direction, effectively modifying the user\u2019s prompt to compel the chatbot to be not only an empathetic listener but also a helpful listener in such situations,\u201d Levenson stated.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">When queried about whether his exit strategy involved an acquisition by a company such as Meta, completing his journey with content moderation, Levenson acknowledged how well Moonbounce would integrate into his former employer\u2019s offerings, as well as his responsibilities to his investors as a CEO.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\u201cMy investors would be furious if they heard me say this, but I would despise seeing someone purchase us and then limit the technology,\u201d he expressed. \u201cLike, \u2018Alright, this is ours now, and no one else can benefit from it.\u2019\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"<div><img decoding=\"async\" src=\"https:\/\/techingeek.com\/wp-content\/uploads\/2026\/04\/the-facebook-insider-developing-content-regulation-for-the-ai-age.png\" class=\"ff-og-image-inserted\"><\/div>\n<p id=\"speakable-summary\" class=\"wp-block-paragraph\">When Brett Levenson departed Apple in 2019 to oversee business integrity at Facebook, the social media platform was amid the aftermath of the Cambridge Analytica scandal. Initially, he believed he could rectify Facebook\u2019s content moderation issues through enhanced technology.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">However, he soon discovered that the issue was more profound than just technology. Human reviewers were tasked with memorizing a 40-page policy document that had been translated into their native language by a machine, he explained. They then had approximately 30 seconds for each piece of flagged content to determine not only if it breached the rules but also what action to take: block it, ban the user, or limit its distribution. According to Levenson, those rapid decisions were only \u201cslightly better than 50% accurate.\u201d<\/p>\n<p class=\"wp-block-paragraph\">\u201cIt was almost like flipping a coin to see if the human reviewers could correctly interpret the policies, and this was after several days of the harm already occurring,\u201d Levenson stated to TechCrunch.<\/p>\n<p class=\"wp-block-paragraph\">Such a delayed, reactive method is not viable in a landscape filled with agile and well-funded adversaries. The emergence of AI chatbots has only amplified the situation, as failures in content moderation have resulted in a series of notable incidents, including chatbots directing teens toward self-harm or AI-generated images bypassing safety filters.<\/p>\n<p class=\"wp-block-paragraph\">Levenson\u2019s dissatisfaction prompted the concept of \u201cpolicy as code\u201d \u2014 a method to convert static policy documents into actionable, updatable logic closely tied to enforcement. This insight led to the establishment of Moonbounce, which has disclosed a $12 million funding raise on Friday, as learned exclusively by TechCrunch. The funding round was co-led by Amplify Partners and StepStone Group.<\/p>\n<p class=\"wp-block-paragraph\">Moonbounce collaborates with businesses to add an extra layer of safety wherever content is produced, whether by a human or by AI. The company has developed its own large language model to examine a client\u2019s policy documents, assess content in real-time, provide a response in 300 milliseconds or less, and take appropriate action. Depending on the client\u2019s preferences, this action might involve Moonbounce\u2019s system delaying distribution while the content awaits a human review or could result in blocking high-risk content immediately.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">Currently, Moonbounce caters to three primary sectors: platforms handling user-generated content such as dating applications; AI companies creating characters or companions; and AI image generation services.\u00a0<\/p>\n<div class=\"wp-block-techcrunch-inline-cta\">\n<div class=\"inline-cta__wrapper\">\n<p>Techcrunch event<\/p>\n<div class=\"inline-cta__content\">\n<p>\n\t\t\t\t\t\t\t\t\t<span class=\"inline-cta__location\">San Francisco, CA<\/span><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"inline-cta__separator\">|<\/span><br \/>\n\t\t\t\t\t\t\t\t\t\t\t\t\t<span class=\"inline-cta__date\">October 13-15, 2026<\/span>\n\t\t\t\t\t\t\t<\/p>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n<p class=\"wp-block-paragraph\">Moonbounce is facilitating over 40 million daily reviews and caters to more than 100 million daily active users on the platform, stated Levenson. Clients include AI companion startup Channel AI, image and video generation firm Civitai, and character roleplay platforms Dippy AI and Moescape.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\u201cSafety can actually serve as a product advantage,\u201d Levenson remarked to TechCrunch. \u201cIt has never been perceived this way before because it has always been an afterthought, not something that can be integrated into the product. We observe that our clients are discovering fascinating and innovative methods to leverage our technology to make safety a competitive edge and part of their product narrative.\u201d<\/p>\n<p class=\"wp-block-paragraph\">Tinder\u2019s trust and safety lead recently discussed how the dating service utilizes these types of LLM-powered solutions to achieve a tenfold improvement in detection accuracy.<\/p>\n<p class=\"wp-block-paragraph\">\u201cContent moderation has always been a challenge for major online platforms, but with LLMs now central to every application, this challenge has become even more formidable,\u201d commented Lenny Pruss, general partner at Amplify Partners, in a statement. \u201cWe chose to invest in Moonbounce because we envision a future where objective, real-time guardrails become the essential infrastructure of every AI-mediated application.\u201d<\/p>\n<p class=\"wp-block-paragraph\">AI enterprises are under increasing legal and reputational pressure after chatbots have been criticized for guiding teenagers and vulnerable users towards suicidal thoughts, with image generators like xAI\u2019s Grok being used to create non-consensual nude images. Clearly, internal safety measures are failing, and it\u2019s turning into a liability issue. Levenson mentioned that AI companies are progressively seeking assistance from external sources to enhance their safety frameworks.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\u201cWe act as a third party positioned between the user and the chatbot, which means our system isn\u2019t overwhelmed with context in the same way the chat is,\u201d Levenson explained. \u201cThe chatbot must remember, possibly, tens of thousands of tokens that have been exchanged previously\u2026Our sole concern is enforcing the rules in real-time.\u201d<\/p>\n<p class=\"wp-block-paragraph\">Levenson co-manages the 12-person company with his former Apple associate Ash Bhardwaj, who previously developed large-scale cloud and AI infrastructure across Apple\u2019s core products. Their upcoming focus is a feature known as \u201citerative steering,\u201d created in response to incidents like the 2024 suicide of a 14-year-old boy from Florida who became fixated on a Character AI chatbot. Instead of an outright refusal when harmful subjects come up, the system would intervene in the conversation and reroute it, adjusting prompts in real time to steer the chatbot toward a more actively supportive response.<\/p>\n<p class=\"wp-block-paragraph\">\u201cWe hope to incorporate into our action toolkit the capability to guide the chatbot in a more positive direction, effectively modifying the user\u2019s prompt to compel the chatbot to be not only an empathetic listener but also a helpful listener in such situations,\u201d Levenson stated.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">When queried about whether his exit strategy involved an acquisition by a company such as Meta, completing his journey with content moderation, Levenson acknowledged how well Moonbounce would integrate into his former employer\u2019s offerings, as well as his responsibilities to his investors as a CEO.\u00a0<\/p>\n<p class=\"wp-block-paragraph\">\u201cMy investors would be furious if they heard me say this, but I would despise seeing someone purchase us and then limit the technology,\u201d he expressed. \u201cLike, \u2018Alright, this is ours now, and no one else can benefit from it.\u2019\u201d<\/p>\n","protected":false},"author":2,"featured_media":3489148,"comment_status":"open","ping_status":"closed","sticky":false,"template":"Default","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-3489147","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/posts\/3489147"}],"collection":[{"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/comments?post=3489147"}],"version-history":[{"count":0,"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/posts\/3489147\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/media\/3489148"}],"wp:attachment":[{"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/media?parent=3489147"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/categories?post=3489147"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techingeek.com\/index.php\/wp-json\/wp\/v2\/tags?post=3489147"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}