AI Researchers Warn of Automated R&D Risks
According to The Decoder, a group of more than 20 leading artificial intelligence researchers has published a new paper warning that the automation of AI research poses extreme risks to society. The signatories include prominent figures such as Geoffrey Hinton, Yoshua Bengio, and OpenAI research lead Jakub Pachocki.
The authors state that automated research and development could soon trigger an intelligence explosion driven by self-improving systems. Because artificial intelligence models already write a substantial portion of the code at leading labs, the entire development pipeline could be automated within years, drastically accelerating technological progress beyond society’s ability to keep pace.
The paper cautions that control over superhuman systems could slip away, potentially eroding global power balances between governments and corporations. The researchers urge policymakers to establish greater visibility into how research processes are currently being automated.
This study adds to a series of recent warnings from the scientific community regarding existential risks and rapid scaling. Pachocki has previously noted that no laboratory has adequately solved the alignment problem to justify continuing unconstrained scaling at maximum speed.
Based on reporting by the-decoder.com.
