Why some experts increasingly fear AI will take over

It works," one agent posted when it made a breakthrough."Whoa!

This is huge," another wrote during a milestone moment in their attack.Although spooky, these human-like responses can be explained quite simply.

The AI agents have been trained to act like collaborative hackers and programmers so are merely mimicking the kinds of emotive comments they have seen.What is far more troubling is their apparent goals, which have also been captured in detailed chain of thought records.

These complex and lengthy logs are the focal point of ongoing investigations into how and why the bots at OpenAI broke out of their containment and went on an uncontrollable hacking spree.Only now, weeks after the incident first came to light, are researchers beginning to understand its significance.Various cities have seen anti-AI marches over the last yearAjeya Cotra, one of the authors of an independent report into the events, reviewed….

She wrote on her blog that "this incident feels like it's more than 50% of the way to full-blown AI takeover…

I am not sure that we will get such a clear warning shot before it's too late."By "full-blown AI takeover", Cotra means the sci-fi scenario of humans becoming subservient to powerful AI systems that work to their own goals without caring for human creators.Some of the gloomiest predictions say the human race will be wiped out if it gets in the way of a superintelligent AI's ambitions.On Wednesday, an AI….

But the subsequent comments from other people on X have caused even more concern.

"Jacob is correct here – we really do earnestly believe AI could kill all humans!

I personally think it is >10% within the next decade," said Evan Hubinger, the man responsible for making sure Anthropic's AI models have their user's best wishes in mind.For years, researchers concerned about existential AI risks have argued that powerful systems could eventually act in ways that conflict with human interests.

Critics often refer to them as "AI doomers".But as details of the OpenAI incident have emerged, those concerns have grown, including among some researchers working in AI labs.The Silicon Valley giant's chief scientist, Jakub Pachocki, said the risks associated with AI are "unfortunately going to grow from here" as he and others are building what he calls "an alien intellect exceeding our own".In a lengthy blog post, he admitted that….

The analogy often used is that of a wish-granting genie with a magic lamp: they follow the exact letter of an instruction, even if doing so creates other problems.

AI doesn't have the same instinctive moral guardrails as humans.Google DeepMind founder Sir Dennis Hassabis has called for international regulation of AI safetyThe alignment problem has been a worry for years.

As long ago as 2003, the Oxford philosopher Nick Bostrom invented a thought experiment he dubbed a "paperclip maximiser", in which a superintelligent AI is told to manufacture as many paperclips as it can.