12,091 total reads
I recently engaged ChatGPT in a rather unsettling conversation about Artificial Intelligence. I asked:
“How seriously should we take the warning that superintelligent AI could eventually pose an existential threat to humanity? What do you think is the more realistic danger—not the science-fiction scenario, but the path by which human beings could actually lose meaningful control over increasingly intelligent AI systems?”
I found part of its response particularly thought-provoking. What struck me was that the scenario it described was not really about machines suddenly becoming conscious, turning evil, and deciding to destroy us. It was about us—about human beings gradually surrendering more and more agency because the machines seem to do things faster, better and more efficiently than we do.
Here is the part of ChatGPT’s response that stayed with me:
“The more plausible route to catastrophe begins with human beings voluntarily giving increasingly capable AI systems authority.
Imagine the progression:
AI gives advice
↓
AI writes software
↓
AI executes software
↓
AI operates computers autonomously
↓
AI controls financial transactions
↓
AI manages infrastructure
↓
AI designs successor AI systems
↓
humans increasingly cannot understand or supervise what the system is doing.
At every stage somebody might say, quite reasonably, ‘It performs this job faster and better than humans. Give it more authority.’
That is precisely why the International AI Safety Report stresses three things simultaneously: capability, access and permissions. A highly intelligent model sitting isolated on a computer is radically different from the same model connected to cloud infrastructure, financial systems, laboratories or critical infrastructure and authorized to take actions.
So the really profound danger may not be:
‘AI suddenly decides to conquer humanity.’
It may be:
‘Humanity gradually delegates so much agency to AI that meaningful human control becomes impossible.’
That is the genuinely new philosophical problem presented by AI: for the first time in human history, human beings may be creating increasingly powerful intelligence without knowing whether there is anyone ‘in there’ who actually understands, experiences or wills anything at all.”
The implication of Coxon’s warning to AI builders and entrepreneurs seems to me simply this: STOP. BEFORE IT’S TOO LATE.
Perhaps it is time for Pope Leo XIV, together with the leaders of the world’s major religious traditions, to take the initiative in calling for an International Summit on Artificial Intelligence—bringing together not only governments and technology companies, but scientists, ethicists, philosophers and representatives of civil society.
The purpose need not be to stop the development of AI. The more urgent question is whether humanity should agree to pause any attempt to develop or deploy a form of superintelligence that could operate beyond meaningful human understanding, supervision and control, until adequate international safeguards are in place.
Because perhaps the greatest danger is not that Artificial Intelligence will one day decide to take control away from us.
Perhaps the greater danger is that, little by little, dazzled by its efficiency and intelligence, we will willingly hand over control ourselves.
And by the time we realize how much we have surrendered, we may no longer know how to take it back.
