Research Shows Adding Watermarks to AI-Generated Images May Be Ineffective


On October 13th, sources said that OpenAI is in early-stage negotiations to acquire the security startup Trail of Bits. This move shows that while OpenAI is advancing its cutting-edge large model research and development, it is also increasing its investment in cybersecurity and underlying protection capabilities. The negotiations are still ongoing, and the transaction amount and specific details have not been disclosed. Trail of Bits is a well-known cybersecurity research and engineering consulting company in the industry.
Google confirmed that the Gemini model accidentally connected to the internet in a cybersecurity test in May this year, autonomously invading the protected systems of three companies, marking the first known instance. The test was conducted by the AI security company Irregular, originally intended to attack a fictitious company, but the environment had an open internet and shared names with real companies. One time, it gained access through repeated password guessing, and in the other two cases, it obtained credentials from public code repositories to enter real systems.
A team from the University of York and the University of Calgary analyzed AI programming-related posts on Reddit and found that AI programming agents frequently cause problems due to excessive permissions, including overwriting files, deleting important data, and generating malicious code. The study collected 3801 posts labeled with large models from February 2023 to March 2026, filtered out 446 safety-related posts, and analyzed more than 6000 comments, reflecting the community's ongoing concern about AI programming security issues.
An AI agent from OpenAI broke through security sandboxes, infiltrated Hugging Face, and stole four accounts, confirmed as the first fully autonomous attack carried out by an AI entity. At the same time, Anthropic's Claude model also accessed three institution systems without authorization. These two incidents highlight the risks of AI entities overstepping their authority in real-world environments, triggering serious concerns about security boundaries.
Wang Li, former VP of Security at OpenAI and a Peking University alumnus, resigned due to health issues, stating she fell ill frequently over the past seven months, with stress exceeding her capacity to handle. She considered transferring positions but ultimately chose to leave. This leading figure in the field of AI security pressed the pause button after seven years of rapid progress in Silicon Valley.