OpenAI Chief Scientist Warns of Rapid AI Self-Improvement Risks

OpenAI Chief Scientist Jakub Pachocki published an article on September 6, 2026, arguing that reasoning language models have become part of a fast-growing economic sector and are beginning to push scientific boundaries. According to ChainCatcher, he said these systems can operate computers and graphical interfaces, collaborate with people and other AI systems, conduct research projects, and are already changing the computer security landscape while creating new risks. Pachocki said current progress could continue toward recursive self-improvement based on internal results, and that systems may see equally large or larger capability jumps in the coming years while increasingly driving their own development. He called for extreme caution, saying no one is prepared for the consequences of rapidly rising machine intelligence, and said OpenAI will continue pursuing technical solutions for alignment and monitoring, building defensive systems, and stopping further scaling if necessary, while also saying broader intervention is needed. He said machine intelligence is driven mainly by increases in compute and that AI is more like growth than design. He distinguished between goal alignment and value alignment, saying generalization remains the core challenge. Pachocki said GPT-6 Astra is significantly better aligned than GPT-5.6 Sol, but more progress is still needed. On monitoring, he said OpenAI is mainly relying on chain-of-thought monitoring, though evaluations show dependence on this capability is gradually weakening. He also said models are becoming superhuman at breaking into computer systems, and that there is now a narrow window to significantly strengthen critical system security using the best available models.