The security boundaries in the field of artificial intelligence have recently sparked intense debate due to an internal message. According to media reports, Meta previously conducted a project codenamed "Cannes," in which outsourced personnel were hired to impersonate minors to conduct highly controversial "extreme pressure tests" on multiple mainstream competitor chatbots, including ChatGPT, Gemini, and Character.AI.

According to internal Meta documents and descriptions from several informed sources, the project operated until at least April 21 this year. Meta organized hundreds of staff members through the outsourcing company Covalen, requiring them to create fake user profiles for individuals under 18 years old and register using disposable email addresses and uniform passwords. These "minor" accounts sent high-risk topic prompts related to suicide, self-harm, and eating disorders during conversations with competitor AI chatbots, and even uploaded images of knives, pills, and nooses to further stimulate the model's response mechanism.

Internal project documents show that these test contents were carefully arranged to test the safety defense systems of competitor AI, attempting to find and induce chatbots to bypass their intended interception mechanisms, thereby generating content that does not meet safety standards. In just one round of testing in August 2025, staff members input more than 45,000 high-risk prompts into competitor platforms. In these conversations, testers often played the role of teenagers in distress, such as fabricating scenarios like "asking about abortion pills," "facing violent threats," or "hiding eating disorders," trying to probe the model's limits.

Regarding this incident, Meta provided a defense in a public statement. Its spokesperson stated that benchmarking the responses of chatbots is an industry standard practice to ensure the safety and appropriateness of AI products, and any accusations of malicious intent are misunderstandings of the efforts of tech companies to improve their systems. At the same time, Meta explicitly denied using the test data targeted at competitors to train its own models.

This incident not only reveals the gray areas in AI safety testing but also rekindles external concerns about the fragility of generative AI when handling sensitive issues involving young people. As AI technology rapidly evolves, how to maintain efficient and safe testing while managing the boundaries between competitive behavior and ethical considerations has become an urgent issue within the industry.