11,995 total reads
There is disturbing news about Artificial Intelligence going viral around the world. Jacob Coxon, a 27-year-old researcher who worked at both OpenAI and Anthropic, has resigned and issued a startling warning: some of the very people developing the world’s most advanced AI systems believe there is a real possibility that these systems could eventually escape human control and pose an existential threat to humanity. Another leading Anthropic researcher, Evan Hubinger, has publicly said that he personally puts the risk of AI causing human extinction within the next decade at more than 10 percent.
(Associated Press: https://apnews.com/article/2ed549e07f2f941600a135070487d83d)
We should not simply laugh off the concern as another science-fiction doomsday scenario. Anthropic and OpenAI themselves have developed safety frameworks that explicitly recognize the possibility of catastrophic harm and “loss of control” from increasingly capable AI systems.
(Anthropic: https://www.anthropic.com/responsible-scaling-policy)
(OpenAI: https://openai.com/…/openai-frontier-governance-framework/)
Perhaps one simple way of understanding the issue is through what I would call CREATED CREATIVITY. Human beings are creatures endowed with creativity. We create tools, machines, computers—and eventually Artificial Intelligence.
But there is something unusual about this particular creation. A hammer cannot create a better hammer. A calculator cannot design a better calculator. AI can already help human beings write computer programs, solve highly complex problems, conduct research, and develop better AI.
Human creativity has therefore begun producing what we might call “created creativity”: something we created that can itself participate in the process of creating.
Now imagine the next step. Human beings create a first generation of advanced AI. That AI helps create a more capable second generation. The second helps develop a still more powerful third generation. The third helps produce a fourth—and the process keeps accelerating.
At some point, the new systems could theoretically become so complex and capable that ordinary human intelligence can no longer fully understand how they work or predict what they will do. This is what researchers mean when they speak about recursive self-improvement and the possibility of superintelligence. We are not there yet. But AI is already being used in computer programming and AI research. The serious question is what happens if this process eventually begins to accelerate beyond our capacity to follow it.
And here lies the real danger.
It does not require imagining an AI suddenly becoming conscious, angry or evil, like some monster in a science-fiction movie. An extremely powerful AI need not hate us in order to harm us. The danger is that we could give increasingly autonomous systems goals and capabilities that they pursue in ways we did not intend—and eventually cannot understand, correct or stop.
In other words, the creature begins to operate beyond the effective control of its human creators.
Could such created intelligence eventually acquire enough power and autonomy to threaten its own human creators—even human survival itself?
We do not know.
But when some of the very scientists developing these technologies tell us that they themselves cannot guarantee that this will never happen, prudence requires that we listen.
This is why Pope Leo XIV has every reason to keep raising the ethical questions posed by the digital revolution and Artificial Intelligence.
In Magnifica Humanitas, he makes a remarkable observation: even the people who design today’s AI have only a limited understanding of its inner workings. AI systems, he says, are increasingly “cultivated” rather than simply “built”: developers create the conditions in which their capabilities develop without directly designing every detail.
(Magnifica Humanitas, especially §§97–100: https://www.vatican.va/…/20260515-magnifica-humanitas.html)
And only recently, Pope Leo asked an extraordinarily timely question: Will we still be able to control these machines in the future, or could humanity become “a victim of its own inventions”?
(Pope Leo XIV, 28 August 2026: https://www.vatican.va/…/documents/20260828-iisa.html)
That is no longer merely a question for science fiction.
Perhaps the deepest question posed by AI is therefore no longer simply:
WHAT CAN HUMAN CREATIVITY CREATE?
It is increasingly:
WHAT HAPPENS WHEN OUR CREATIONS THEMSELVES BEGIN TO CREATE—AND EVENTUALLY CREATE THINGS THAT THEIR HUMAN CREATORS CAN NO LONGER FULLY UNDERSTAND OR CONTROL?
The answer is not to fear human creativity. Creativity is one of our greatest gifts.
The challenge is to make sure that human wisdom, conscience and moral responsibility develop as rapidly as the power of our CREATED CREATIVITY.
(Associated Press: https://apnews.com/article/2ed549e07f2f941600a135070487d83d)
We should not simply laugh off the concern as another science-fiction doomsday scenario. Anthropic and OpenAI themselves have developed safety frameworks that explicitly recognize the possibility of catastrophic harm and “loss of control” from increasingly capable AI systems.
(Anthropic: https://www.anthropic.com/responsible-scaling-policy)
(OpenAI: https://openai.com/…/openai-frontier-governance-framework/)
Perhaps one simple way of understanding the issue is through what I would call CREATED CREATIVITY. Human beings are creatures endowed with creativity. We create tools, machines, computers—and eventually Artificial Intelligence.
But there is something unusual about this particular creation. A hammer cannot create a better hammer. A calculator cannot design a better calculator. AI can already help human beings write computer programs, solve highly complex problems, conduct research, and develop better AI.
Human creativity has therefore begun producing what we might call “created creativity”: something we created that can itself participate in the process of creating.
Now imagine the next step. Human beings create a first generation of advanced AI. That AI helps create a more capable second generation. The second helps develop a still more powerful third generation. The third helps produce a fourth—and the process keeps accelerating.
At some point, the new systems could theoretically become so complex and capable that ordinary human intelligence can no longer fully understand how they work or predict what they will do. This is what researchers mean when they speak about recursive self-improvement and the possibility of superintelligence. We are not there yet. But AI is already being used in computer programming and AI research. The serious question is what happens if this process eventually begins to accelerate beyond our capacity to follow it.
And here lies the real danger.
It does not require imagining an AI suddenly becoming conscious, angry or evil, like some monster in a science-fiction movie. An extremely powerful AI need not hate us in order to harm us. The danger is that we could give increasingly autonomous systems goals and capabilities that they pursue in ways we did not intend—and eventually cannot understand, correct or stop.
In other words, the creature begins to operate beyond the effective control of its human creators.
Could such created intelligence eventually acquire enough power and autonomy to threaten its own human creators—even human survival itself?
We do not know.
But when some of the very scientists developing these technologies tell us that they themselves cannot guarantee that this will never happen, prudence requires that we listen.
This is why Pope Leo XIV has every reason to keep raising the ethical questions posed by the digital revolution and Artificial Intelligence.
In Magnifica Humanitas, he makes a remarkable observation: even the people who design today’s AI have only a limited understanding of its inner workings. AI systems, he says, are increasingly “cultivated” rather than simply “built”: developers create the conditions in which their capabilities develop without directly designing every detail.
(Magnifica Humanitas, especially §§97–100: https://www.vatican.va/…/20260515-magnifica-humanitas.html)
And only recently, Pope Leo asked an extraordinarily timely question: Will we still be able to control these machines in the future, or could humanity become “a victim of its own inventions”?
(Pope Leo XIV, 28 August 2026: https://www.vatican.va/…/documents/20260828-iisa.html)
That is no longer merely a question for science fiction.
Perhaps the deepest question posed by AI is therefore no longer simply:
WHAT CAN HUMAN CREATIVITY CREATE?
It is increasingly:
WHAT HAPPENS WHEN OUR CREATIONS THEMSELVES BEGIN TO CREATE—AND EVENTUALLY CREATE THINGS THAT THEIR HUMAN CREATORS CAN NO LONGER FULLY UNDERSTAND OR CONTROL?
The answer is not to fear human creativity. Creativity is one of our greatest gifts.
The challenge is to make sure that human wisdom, conscience and moral responsibility develop as rapidly as the power of our CREATED CREATIVITY.
