Anthropic, OpenAI Researchers Warn Recursive Self-Improvement Could Accelerate AI Risks

According to CNBC, researchers at Anthropic and OpenAI warned this week that recursive self-improvement, or RSI, could rapidly increase AI capabilities and reduce human control over how new models are trained. Evan Hubinger, an alignment lead at Anthropic, said on X that he thinks there is more than a 10% chance AI could kill all humans within the next decade, while OpenAI Chief Scientist Jakub Pachocki said he was concerned that no one was prepared for the consequences of a continued rapid rise in machine intelligence. Anthropic said in June that its internal data showed Claude was accelerating AI development and that the process was happening faster than expected. The company also said in an August blog post that its engineers now ship eight times as much code per quarter as they did between 2021 and 2025.