AI researchers warn of self-improving systems race

Current and former researchers at leading AI labs are warning that companies are racing to build self-improving AI systems faster than safety measures can keep up.
In video testimonials gathered by the nonprofit Palisade Research and reported by Reuters, OpenAI and Google DeepMind researchers said competitive pressure is pushing labs towards increasingly autonomous systems before the risks are understood.
A separate paper by more than 20 researchers, including Geoffrey Hinton, Yoshua Bengio and OpenAI research chief Jakub Pachocki, warns that automating AI research could compress years of progress into months, leaving little time for oversight.
DeepMind researcher Neel Nanda put the chance of AI-caused human extinction at at least 10 per cent, calling it “ridiculously high”. Anthropic chief executive Dario Amodei has urged the industry to “pace the frontier”, and OpenAI’s Sam Altman agreed. But critics say the labs keep launching more powerful models while asking others to slow down.
Leave a Reply