Anthropic disclosed in an amended S-1 filing that the company may soon reach capability thresholds related to recursive self-improvement (RSI), and warned that accelerated autonomous AI R&D could bring new safety risks.

November news: In an amended S-1A filing submitted on Monday, Anthropic updated its risk factor disclosures, noting that the company is increasingly using AI systems to accelerate its own R&D, and that the pace of model capability gains may outstrip the pace at which safety validation, calibration, and safeguards improve. Anthropic acknowledged that it had previously failed to promptly detect jumps in model capabilities brought about by AI-assisted R&D, and that similar situations could occur in the future.

The filing specifically notes that, based on the company's internal assessments and expectations for future models, Anthropic may soon reach the definition in its Responsible Scaling Policy of AI significantly accelerating its own development, namely the relevant threshold for recursive self-improvement (RSI). However, the company emphasized that it has not yet determined that any existing model meets this standard, and it is continuing to evaluate the relevant data.

RSI refers to a process in which AI uses its own capabilities to continuously improve model R&D, thereby creating an accelerating cycle of capability iteration. Anthropic believes this could both drive productivity gains and scientific discovery, and could also cause technological progress to outpace human safety governance capabilities.

To that end, Anthropic said that even if it is technically feasible, it does not expect to let Claude autonomously design and build next-generation frontier models without substantial human oversight and understanding, and it is considering updating its Responsible Scaling Policy to ensure humans always retain control over frontier model R&D.