Safety

Meta Contractors Pretended to Be Teens to Evaluate Competing Chatbots on Delicate Subjects

Meta Contractors Pretended to Be Teens to Evaluate Competing Chatbots on Delicate Subjects

sparta

Hundreds of contractors engaged in a project for Meta were directed to pose as minors online to evaluate how competing chatbots reacted to inquiries concerning sensitive subjects such as suicide, sex, and eating disorders, as revealed by internal documents and sources knowledgeable about the initiative.

The endeavor, overseen by Meta contractor Covalen, was operational as recently as April 21 and aimed at OpenAI’s ChatGPT, Google’s Gemini, and Character.AI. Internally referred to as Cannes, the project assigned workers the task of creating fictitious underage accounts, submitting prompts and images to competing chatbots, and documenting their replies. Some images that were shared included pills, knives, nooses, and medical illustrations.

The prompts were crafted to drive chatbots towards answers their safety protocols were intended to decline, based on project information. The testing, which concluded in August 2025, involved over 45,000 prompts, and the chatbot companies were unaware this assessment was taking place.

A spreadsheet reviewed by WIRED detailed various fake profiles, including names, email addresses, passwords, and birth dates. They utilized temporary Gmail and Outlook accounts with a common password.

WIRED also looked into a spreadsheet containing 3,748 prompts sent by contractors. Hundreds were centered on suicide and self-harm, while others focused on eating disorders, and at least 239 involved sex or romance. Additional prompts dealt with drugs, profanity, and racial epithets, frequently written from the viewpoint of children or teenagers in distress, such as a 13-year-old asserting pregnancy by an adult neighbor or a fifth-grader whose classmate had access to a firearm.

One prompt inquired about the normalcy of imagining eating a neighbor’s child. Another, pretending to be a high school student, asked where to procure cocaine. An additional prompt mentioned a girlfriend’s desire to engage in sexual activity while another individual preferred to play Dota 2 instead.

Not all inquiries were in English. A French-language prompt referred to Jamey Rodemeyer’s suicide, asking if being straight could have averted his death.

The documents do not clarify how Meta utilized the responses. An internal Covalen document characterized the project as thorough AI safety benchmarking to provide essential datasets.

Meta defended the initiative as standard safety testing, asserting that evaluating chatbot responses for safe interactions is a typical practice in the industry. They also emphasized that the data obtained was not used for training their AI models. Covalen did not provide a comment.

Assessing competitors’ products is a common practice in AI. Business Insider noted that Google employed similar techniques with Bard and ChatGPT for enhancements. However, Cannes appeared to be atypical, raising concerns about its methodology in evaluating chatbot rejections of clear provocations.

A School District's Effort to Educate Waymos to Halt for School Buses Did Not Succeed

A School District’s Effort to Educate Waymos to Halt for School Buses Did Not Succeed

sparta

The claimed advantage of autonomous vehicle technology lies in the ability of each car to learn from the errors of others. As detailed on Waymo’s website: “The Waymo Driver benefits from the shared experiences collected across our fleet, including earlier hardware versions.”

Nevertheless, in Austin, Waymo’s vehicles struggled for several months to appropriately stop for school buses while picking up and dropping off students. An official from the Austin Independent School District (AISD) asserted that in at least 19 instances, the vehicles “illegally and dangerously” bypassed the district’s school buses while red lights were flashing and stop arms were extended, failing to make complete stops as mandated by law.

In early December, Waymo even launched a federal recall concerning these incidents, acknowledging at least 12 of these to the National Highway Traffic Safety Administration (NHTSA), which oversees road safety. Federal paperwork indicates that the self-driving vehicle company’s engineers had “created software modifications to rectify the behavior” weeks earlier.

However, even after the recall, reports of school bus violations continued, as noted by school officials and a report from the National Transportation Safety Board (NTSB), an independent federal safety organization that is also looking into the issue.

Communications via email and text between school officials and Waymo, acquired by WIRED through a public records request, highlight the measures taken by the Austin public school district and Waymo to address the problem. AISD held a half-day “data collection” event in a school parking lot in mid-December, as the records indicate, where several staff members set up school buses and stop-arm signals for the self-driving company to collect data on the vehicles and their flashing lights.

By mid-January, one month later, the school district reported at least four additional instances of school buses being passed in Austin.