In this conversation, The Republic editor Bill Rivers and Meritocracy Fellow Jaivir Singh sit down with John-Clark Levin, Research Lead at Kurzweil Technologies, to discuss how the Catholic Church, through its moral authority and global influence, is uniquely positioned to shape both societal ethics and international policy on Artificial General Intelligence (AGI).
Bill Rivers (BR): This a timely discussion. The Republic explores consequential issues in technology, national security policy, and international affairs. In addition to its theological and moral teaching role, the Catholic Church really sits in the center of that Venn Diagram.
You’ve written that the Church needs to engage more robustly on AGI. Before we dive into why and how, let’s define our terms. How do you define AGI?
John-Clark Levin (JCL): Artificial General Intelligence is AI that is at least at the level of the median educated human at all empirically measurable cognitive tasks. This says nothing of ensoulment. It speaks only to things we can measure.
BR: Are we there already?
JCL: We are not. One good metric for assessing this is the Abstraction and Reasoning Corpus or ARC-AGI benchmark. This is essentially an IQ test for AI. It contains fairly simple visual patterns and asks AI from just three examples to infer a fourth completion. It has been improving exponentially over the past few years, but humans are still better at it than the best AI. That’s changing fast, though. My best guess is AGI arrives around the end of this decade.
BR: The Catholic Church as an institution claims a monopoly on the truth. Said differently, the Church holds that other religions can participate in the truth, but that the Catholic Church participates most fully. What is the bearing of AGI to an institution making this claim?
JCL: Almost limitless. For example, the Church is thinking seriously about how to ensure democratic control of powerful AI systems. Pope Francis said in 2023 the voices of all stakeholders should be weighed, including “the poor, the powerless and others who often go unheard in global decision-making processes.”
In early 2025, the Church also released a document called Antiqua et Nova (“Old and New”), formalizing his insistence that decision-making on AI be shared broadly. The Church is clearly worried about AI’s potential to advance narrow interests at the expense of the common good.
BR: Where is your focus on AGI vis-a-vis the Catholic Church?
JCL: There are three pillars around which I argue the Church is not yet taking AGI seriously enough — catastrophic risk, economic impact, and epistemic impacts.
Catastrophic risk means risk not just to individual people or groups, but entire nations, civilizations, the human species. The chatbots and weak agents of today pose essentially zero catastrophic risk. But once AI is broadly superhuman, the scope of potential harms expands dramatically.
The Church is already thinking seriously about sub-catastrophic risks, like algorithmic bias or lethal autonomous weapons. Real issues, but little chance they’ll destroy civilization.
BR: Can you give us an example of catastrophic risk?
JCL: Bio-risk scenarios. AI could design a DNA or RNA sequence for a virus deadlier than those that might arise in nature. In nature, viruses face a strong evolutionary trade-off between transmissibility and lethality. Some are very contagious, like SARS-CoV-2, which initially killed around 1% of people who caught it. Others, like Ebola, have evolved to be very deadly with something like a 20-90% fatality rate but are actually not all that contagious.
In very rare cases, a pathogen combines both high transmissibility and high lethality. You get something like the 1545 Cocoliztli epidemic, which wiped out 80% of the population of Mexico. Virologists know what functional boxes a virus would need to check to kill on that scale or even worse. The only missing ingredient is AI smart enough to spit out a DNA or RNA sequence that actually checks those boxes.
BR: Reminds me of the “Captain Trips” super flu in Stephen King’s novel The Stand.
JCL: Which was developed by the military for geopolitical competition. But AGI bio-risk need not involve human actors trying to create a pandemic virus. AGI trained hurriedly in a race to beat China and deployed without proper safety checks might have a hidden flaw, such that while running university virology lab equipment it suffers the future equivalent of today’s chatbot hallucinations and decides human extinction is morally necessary.
Maybe it reads too much Nietzsche and concludes human life is so miserable we’re better off dead. Or maybe it’s been trained with a deep-seated concern about climate change and realizes CO2 levels would plummet if there weren’t eight billion Homo sapiens.
Note this isn’t a sci-fi robot with glowing red eyes. This would just be AI trying to follow ostensibly helpful principles and making errors in ways not fundamentally dissimilar from how ChatGPT bungles things now.
Jai Singh (JS): But AGI’s potential to upend society isn’t limited to pandemics and sci-fi nightmares, right?
JCL: No.
JS: You mentioned economic impact. This is where most of the practical strands of the AI debate live right now. But that’s also where there’s enormous opportunity for human progress.
JCL: The Church appears to be thinking about AI how it thought about the Internet. In the 1990s and early 2000s, the Church figured the Internet would create a bunch of jobs and destroy a bunch, but basically just nibble around the edges of the labor market, not fundamentally change the economic system or upend the relationship between labor and the human person.
AGI is different. It is likely to destroy hundreds of millions of jobs around the world by the end of next decade, and although many new jobs will be created, probably not enough accommodate all the people who lost theirs.
Society has to figure out what our responsibilities are to people suddenly unable to earn a livelihood through no fault of their own. We need to think through what they’ll do with their time, if they are not working. This is not an intractable problem, but it’s a challenge we must address systematically. The Church’s understanding of human purpose can be a great aid.
JS: Yeah, but don’t we first have to accept your premise that AGI is only a few years out? The greater the time horizon, the greater the likelihood that humans can adapt as needed. It seems like for this to be true, AGI would have to come down the hill so fast that it totally eliminated higher-skill job creation, technical retraining, etc.
JCL: There are now several lines of empirical evidence converging on short AGI timelines. Not just steady exponential gains in computing hardware price-performance, algorithmic efficiency, and compute scale, but also quantitative modeling of how automated coding speeds up AI progress, and direct AI performance metrics like completing long-time horizon tasks. But the labor impacts don’t hinge on whether AGI arrives in 2029 or 2040. Either way, it’s much shorter than a career-length away — compressing the Industrial Revolution’s century-long disruptions into a single generation.
BR: Let’s look at epistemic impacts. How is AI shaping what we know and believe?
JCL: This is under-discussed, and it’s ironic, because it has to do with something ubiquitous, and that’s social media.
Most people today think of AI primarily as chatbots. They don’t realize that even if you’re a ChatGPT power user, you’re probably still mostly engaging with AI via what social media algorithms decide to show in their newsfeeds. Most Americans are on social media two, three, four hours a day. Most of the information we consume is filtered through these AI algorithms that are trained not to enlighten us but to maximize our engagement. Practically, that usually means stoking fear and anger.
JS: But this has been true for years. What makes it so significant that the Church needs to weigh in on it?
JCL: Catholic Social Teaching urges citizens to “take an active role in public life,” but emphasizes each person’s obligation to cultivate a “well-formed conscience” to guide this. It’s hard to do that if you’re guzzling algorithmically targeted ragebait and misinformation morning to night.
Having good moral values isn’t enough. Our moral obligations around public life — who to vote for, whether to protest or disobey an unjust law — often depend on empirical realities of what’s happening in our community.
AI can make our sources less reliable. Deepfakes are now so convincing it’s often impossible to judge with our eyes whether an image or video is real. With AGI, it will likewise be impossible to judge whether someone you meet online is a real person or a bot deployed to radicalize you — yes, you personally.
For the past 15 years, web platforms thwarted most bots with CAPTCHA tests. But as the AI bots have gotten smarter, CAPTCHAs have had to get harder to match, to the point that a lot of humans, especially the elderly or vision-impaired, can’t solve them.
JS: They’re not alone! I struggle with CAPTCHAs all the time.
JCL: So do we all. It reminds me of the apocryphal quote attributed to a park ranger at Yosemite, on the impossibility of making bear-proof trashcans, that there’s substantial overlap in intelligence between the smartest bears and the dumbest tourists. So that kind of capabilities test won’t protect us from AGI disinformation.
JS: Isn’t this just a technical problem with a technical solution? If we can reliably identify AI-generated content, we can flag it.
JCL: I don’t think a technical solution will be sufficient. Brilliant engineers are working all kinds of deepfake watermarking schemes, blockchain authentication, counter-AI AI, and so on. But there are easily accessible tools that exist now, and people don’t bother using them.
If citizens don’t feel a twinge of moral obligation to check their sources, none of that matters. Far too many aren’t practicing a basic level of care about what they believe and then share — with their credibility attached — to friends and family.
In philosophy terms, they have poor “epistemic hygiene.” As AI amplifies the harm of bad epistemic hygiene, the moral obligation to practice good epistemic hygiene grows more urgent.
The challenge for the Church is to deeply understand this new reality. Twenty years ago, when today’s cardinals were in more pastoral roles as priests or junior bishops, social media meant MySpace. If you posted a hateful conspiracy theory, who would see it? Your top eight friends, and maybe a dozen others who deliberately clicked on your profile. There was no mechanism for virality. Morally, the Church saw that as technically wrong, but not a high-stakes issue. Morally more akin to office gossip than inciting a mob. Now you post that same conspiracy on TikTok, and if you catch the algorithm just right, you can go to sleep and wake up the next morning with millions of views. You can cause real-world violence.
These are the stakes of epistemics. If we do not recover that shared moral obligation to each other, our chances of living together peacefully are in serious jeopardy.
BR: And the Church can shape that moral obligation?
JCL: Yes. Pope Leo has both the moral stature and bipartisan credibility, for lack of a better term, to call people back to practicing better epistemic hygiene.
JS: What should the Catholic Church do?
JCL: There are two main pathways Pope Leo XIV can have an impact. The first is an encyclical.
The current pope took his name from Pope Leo XIII, who wrote Rerum Novarum (“Of New Things”) in 1891, which was about how the Church and society should respond to the Industrial Revolution. This became the foundation of Catholic Social Teaching and had a profound impact on how the church thinks about issues like poverty, for example.
From the life of Jesus up until Rerum Novarum, the Church mainly thought about poverty in a person-by-person sense: “You’re hungry, here’s some bread.” After Rerum Novarum, the Church began to think more systemically: “How can we as the body of Christ work together for a more just society such that people will not be poor and hungry in the first place?”
Pope Leo is reported to be writing an encyclical to address anthropological — “humanness-related”— issues, with a significant focus on AI. The hope is that this will be the intellectual heir to Rerum Novarum as a parallel for the AI age. However, if this encyclical focuses overwhelmingly on AI as it exists today, on Chat GPT and the like, it’s likely to be a missed opportunity.
Because if AGI comes within the next decade, as scientists now consider very likely, its impact will be vastly more profound than chatbots. Imagine if Pope Leo XIII had written not on the whole Industrial Revolution in 1891, but merely on the impact of the power loom or the spinning jenny. His encyclical would have missed responding to the truly seismic social changes resulting from more advanced technology like steam engines and electricity.
To meet this moment, we need to take the prospect of AGI seriously now, prospectively. Many powerful decisionmakers still think this is sci-fi. If the Pope says we don’t know it will happen, but it is plausible and we should prepare, that’s a powerful mobilizing signal.
BR: The crux of your argument then is that the Church should speak on AGI holistically as opposed to focusing on capabilities or affected groups or some other sub-categorization.
JCL: Yes. The Church can speak on the moral implications for everyone from scientists and CEOs building these models, to the policymakers who regulate them, to the citizens who use them.
An encyclical is one pathway. The other would be using the diplomatic power and perceived neutrality of the Holy See to mediate the AGI arms race that is unfolding between the United States and China.
When Vice President Vance was in Rome last May for Pope Leo’s inauguration, he spoke to Ross Douthat on his podcast and expressed hope that the Holy Father could play a direct role in mediating AI arms race with China. Those comments may motivate Vatican diplomats who are concerned about AI to say: “The Americans are tentatively onboard. What kind of outreach can we do with the Chinese?”
This outreach could be surprisingly influential with Beijing, because it doesn’t require the atheistic Chinese Communist Party to embrace Pope Leo’s religious authority. The CCP is deeply paranoid — even by standards of authoritarian regimes. The current leadership class around President Xi was coming up in the Party at the time of the 1989 protests and were deeply scarred by Tiananmen Square. They will be terrified if they can be convinced of the scientific case that racing to develop AGI risks it getting out of human control and destabilizing or disempowering their regime. If Pope Leo can bring the Americans back to the table, that could make some degree of arms control a more attractive option.
This wouldn’t be an outright ban on AGI research, but an agreement about red lines —dangerous development practices that both sides should be willing to forswear if they have reasonable assurance their opponents do the same. Not letting AI modify itself without close human oversight would be one example. Verification methods for such a treaty could draw inspiration from Cold War nuclear arms control efforts. Back then, credible, low-trust protocols that relied on scientific instruments to detect violations laid the groundwork for more comprehensive inspection protocols like the Intermediate-Range Nuclear Forces Treaty.
I don’t think Pope Leo can or needs to halt the AGI arms race altogether, but he could lower the temperature just a little bit. That could make sure whoever wins can take enough safety precautions, swinging this from a very bad outcome for humanity to a very good one.
BR: I am only interested in the United States winning that race.
JCL: It’s looking favorable.
JS: I don’t have much knowledge of the Catholic tradition, so let me address some of the core social and engineering consequences of AI. What are the positives of AGI, if any?
JCL: Enormous positives. Curing cancer is a cliché answer, but it is a real prospect. Immunotherapies like CAR-T have been miraculous at saving 40-50% of patients who would otherwise have no hope for certain kinds of cancers. But the reason it doesn’t work for the other half is usually because the cancer manipulates its immediate surroundings, the tumor microenvironment, in ways that blind or confuse the immune system.
Artificial intelligence has a superpower for finding patterns in settings like that and modeling highly complex interactions. But once it also surpasses human general reasoning ability, it would be able to connect the patterns it finds to deep causal analysis. To say: “This cytokine action is what a drug would have to interfere with, and everything else is noise.”
Those same abilities could be applied to fields like materials science to unlock abundant energy and ultra-cheap manufacturing. This would be very good for Americans, but absolutely transformative for the global poor. Many people find this surprising.
There’s a widespread intuition that AGI would be a superpower for the rich that leaves everyone else behind, but that’s the opposite of what we’re seeing so far. Since before ChatGPT, metered AI prices have been getting around 90% cheaper every year. And prices for a constant level of model capability are falling roughly 97% per year. Separately, but related: someone in Rwanda today with no running water but a $16.50 smartphone can go on ChatGPT and connect for free with a smarter model than Donald Trump or Elon Musk could access with all their wealth and power just 90 days ago.
So AGI brings both extremes — both catastrophic risk and transformative hope. And it is on pace to arrive within a number of years small enough, as Nabokov put it, to count on the fingers of one maimed hand.
JS: But the essential fear behind AGI is that we won’t figure out how to make models aligned with human interest. Why should we assume that won’t happen? Why are we doomed to be house cats?
JCL: To be clear, I’m not saying that alignment won’t happen. I’m saying that achieving alignment will require supreme effort by our civilization — of the scale exerted in the Cold War that pulled humanity back from the brink of nuclear annihilation.
AI is no longer crafted by humans in the way software like Microsoft Windows was created, where the engineers design the algorithms themselves and therefore know how the whole system works. Instead, large language models like ChatGPT, Gemini, and Claude are created much more like how a farmer grows a vegetable. The farmer doesn’t need to understand the molecular biology of how seeds germinate. Instead, he can operate at a higher level of abstraction, knowing that if he plants seeds, fertilizes them, and waters them, the laws of nature will make food spring forth from the soil. What happens inside top AI labs is eerily similar.
There’s not one scientist alive who knows how to hand-program all the trillion-plus simulated synaptic connections in the neural network of a large AI model. But labs know how to set up the right conditions for intelligence to grow on its own. The model is “fertilized” with tens of trillions of words of training data, and “watered” by specialized chips doing around a hundred trillion, trillion operations of math to find the hidden patterns in that data. And out comes an AI capable of writing poetry or solving differential equations. This is deeply strange and sci-fi, but it works.
The problem is that when an AI is created this way, its inner workings are a black box, and the human developers do not know all its capabilities and goals. Instead, they try to figure this out experimentally, like biologists studying a creature in a lab. Even though hundreds of people test an AI before release, when it is deployed to hundreds of millions of users poking and prodding at it, they will — not might, will — find failure modes the company didn’t expect.
Labs invest tens of millions of dollars in “guardrails”— extra training processes and technical mechanisms designed to prevent the AI from misbehaving. But an online “jailbreaker” has figured out how to circumvent every major model’s guardrails within 24 hours of release.
So this is fundamentally a scientific problem much more than an ethical one. Yes, there are real and thorny questions to resolve about the values we want AI to have. But the central issue is that we do not yet know how to robustly instill any values in an AI with current techniques.
One promising field is “mechanistic interpretability,” which uses mathematical techniques to look under the hood of AI and map out how all those simulated neural connections actually work. If we could fully map what’s going inside AGI, we could have much better assurance about whether it’s safe. But that research still has a long way to go.
And then there is low-hanging fruit on policy. For example, taking a page from biosafety protocols, AGI research should only be done in data centers that are air-gapped, not connected to the live internet.
BR: Last question. You’ve said authoritarianism also exacerbates risk from AGI. How so?
JCL: Two main ways. First, leaders with authoritarian impulses tend to view the world through a zero-sum lens, predisposing them to risky arms racing rather than cooperative development. And they surround themselves with yes-men, so when they have to make important decisions under high uncertainty, there’s nobody left in the room to tell them hard truths or go against the consensus. This is very dangerous if leaders are making strategic decisions around a race toward AGI, which could potentially include acts of war to stop a rival from winning the race.
Second, if authoritarians gain control of AGI, it would supercharge their capacity to surveil, censor, and repress. Consider China: today, citizens can still outpace human censors through creative workarounds like homophonic character substitutions. But AGI would mean a superhumanly intelligent censor watching everything every citizen writes in real time.
We shouldn’t assume this threat stops at foreign shores. AGI will become the primary platform through which citizens get information, creating enormous incentives for any government, including ours, to influence its outputs. The executive branch already has extraordinary leverage over AI labs through pending antitrust actions and defense contracts. This is a truly bipartisan concern: whatever your politics, you should worry about the next administration having that power.
This means companies serious about civil liberties need to take protective action much earlier in the development process. Audit trails and access controls are fine when the failure mode is a rogue analyst spying on his ex-girlfriend, but if you hand a government — even a democratic one — tools that can be easily reconfigured for mass surveillance, that’s not enough. The focus must shift to identifying worst-case abuse pathways and engineering products to actively resist those abuses.
This will ultimately require gating the most powerful capabilities behind infrastructure that can be monitored and shut off if abused — and building in genuinely independent oversight. That will be a huge challenge from a security, privacy, and classification standpoint. I don’t pretend otherwise. But the alternative is creating tools that can dismantle liberty.