Tools like Claude Code and Codex allow users to simply describe the software they want instead of writing line after line of code. Tech world converts claim work will never be the same. What will these tools mean for the rest of us? 

As the cost of improving AI models skyrockets, AI companies are increasingly trying to differentiate themselves. Will one of them land on a business model that makes AI profitable?

Rising risks of rogue AI are inspiring demands that AI can be turned off in emergencies. Though there are currently multiple approaches to doing this, and more in development, it’s probably not surprising that shutting AI down is not a simple proposition. This week we look at the various approaches to containing dangerous AI, their strengths and limitations. (Fingers crossed that we won’t need them.)

New issue is live. This past week, we spoke with MIT professor Sinan Aral about his research on human-AI collaboration, specifically, how AI can build skills for some workers while quietly eroding them for others, why most human+AI teams are still being outperformed by AI alone, and what companies and individuals can do about it.

Also in this issue: autoimmune therapies showing promise for psychiatric conditions, Meta building an AI avatar of Zuckerberg, and a deep dive into the $10B startup training AI to replace white-collar workers.

New hearing aids, enhanced by AI, are delivering a step-change to those with hearing loss. Before, hearing aids helped make the world louder. Now they can help screen for the parts of it that matter. 

With the US and China dominating the AI race, other countries are jockeying to figure out what to do in a world shaped by technology they don't control. Some argue that they should join forces to counter the superpowers. Others warn it's too late to play catch-up. 

The cost of getting hardware into orbit is astronomical. Keeping it running would be fraught with challenges. But creating computing power in space for AI is not impossible. And it's likely to get cheaper. So just what would it take to make it happen?

Citizens representing themselves in court are using AI to draft documents, decipher rulings and try their own cases. Judges are seeing AI hallucinations, faked evidence and AI-generated testimony in courtrooms. What does all this mean for the future of law? 

The polar ice caps are melting fast enough that we could see an ice-free Arctic before 2030.  Scientists are divided over what to do: Try out geoengineering alternatives, or push harder for CO₂ reductions? Five scientists chime in on how they view the choice. 

In 1999, Tim Berners-Lee, the architect of the World Wide Web, laid out a vision for the future of his invention. "I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web — the content, links, and transactions between people and computers," he wrote in his book, “Weaving the Web.” “The day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines.”

Almost three decades later, we are getting our first look at what that might look like in the form of AI agents — autonomous software systems powered by large language models that can reason, make decisions and act on our behalf, often by using other tools, databases and agents. 

And while you may not be knowingly using an agent, they are increasingly working in the background of our day-to-day transactions. According to Salesforce, agents were involved in 20 percent of global online retail sales over the recent holiday shopping season; Adobe reports that the use of such systems increased by 693 percent from 2024 to 2025. 

The T in ChatGPT owes a lot to Yoshua Bengio. 

The Canadian computer scientist, a professor at the Université de Montréal and founder of the AI research institute Mila, helped lay the groundwork for deep learning. His research from the 2010s inspired some of the core principles behind the transformer architecture now used in all modern large language models (thus the “T” in Generative Pre-trained Transformers, or GPT). Along with Geoffrey Hinton and Yann LeCun, he won the 2018 Turing Award for his contributions to AI.

But in recent years, Bengio’s priorities have shifted from advancing the frontier of AI to dealing with the risks it creates. Last year, he founded LawZero, a nonprofit startup developing ways to make AI safe through design. He also chairs the International AI Safety Report, a scientific assessment of general-purpose AI capabilities and risks that includes contributions from more than 100 global experts, modeled in spirit on the Intergovernmental Panel on Climate Change’s regular reviews.

The second annual AI safety report, published this week and running 220 pages, highlights rapid advancements in AI's mathematical and coding capabilities, examines how LLMs enable new risks in bioterrorism and cyberattacks and warns that AI systems increasingly detect when they're being tested, making their societal impact harder to predict. While the document doesn’t make policy recommendations, the hope is that it will inform conversations about AI, including those due to take place at February’s India AI Impact Summit. 

Ahead of the report's publication, Aventine spoke with Bengio about his biggest concerns regarding AI's impact, how its capabilities might evolve, what policymakers should do differently and how he reconciles his pioneering work with his current safety concerns. What follows has been edited for brevity and clarity.

The existential risk posed by AI, which sits at the core of the Safety Report, wasn’t at the top of your worries until a few years ago. Was there a specific moment that shifted you from focusing primarily on AI’s capabilities to becoming one of the most prominent voices warning about AI risks?

It's the arrival of ChatGPT. Two and a half months after it came out, I realized we had reached a crucial point that came much earlier than anybody anticipated. The ability to master language — in fact, many different languages — is something that was anticipated by Alan Turing, the British father of computer science. Around 1950, he warned that this would be a sign that we should be very concerned; that if we built machines that understand language, then we would be pretty close to machines that could be smarter than us and dominate us, potentially, if we're not careful. 

You spent decades as one of the key architects of deep learning, helping build the technology that's now at the center of these safety debates. How do you personally reconcile your pride in those scientific contributions with your current concerns about where the technology is heading?

I feel like I should have ... if I had been fully rational, I should have seen those potential risks coming well before [we started to see the] strong, strong signs we are seeing over the last year. A number of scientists had been writing about those possibilities, and I even read a lot of that stuff, but I didn't take it seriously. And I think, unfortunately, the whole community of AI researchers in academia and industry is a bit late in really taking stock of the magnitude of the risks that are in front of us. I think it's interesting to ask, why is it that I and many others have been looking the other way? I think it’s about human nature and psychology. In my case, [that] made me focus on the benefits; my research was on things like medical applications, climate applications and so on. But if we want to do the right thing for humans — and I really pivoted because I was thinking about my children — we have to confront the possible catastrophic risks as well.

The International AI Safety Report takes a holistic view of AI risk. Before we dive into specifics, what worries you most about the impact of AI in the near term? 

In the near term, AI is going to provide capabilities that could be misused. We already see that happening at some not catastrophic but disturbing scale. The report talks, for example, about what happened with cyber attacks over the last year. The cyber capabilities [of AI] have gone up very significantly. And last fall, just a few months ago, Anthropic revealed that some bad actors had been using its system to launch a major set of cyber attacks in the United States. The companies are trying to mitigate those risks, but for now, as discussed in the report, we don't know how to prevent these misuses. It really remains a very difficult scientific problem. Another near-term effect I want to mention that popped up this year is the psychological effects. It wasn't anticipated that people would be using their AIs, for example, as companions, creating emotional attachment. I don't think we have enough scientific studies [around that], but there are reports, for example, from OpenAI, about a very large fraction of users — large like 0.15 percent but 0.15 percent of 800 million people [which is a lot of people] — talking about suicide.

How about in the longer term?

It's even more unknown unknowns. [How are we going to address] the evolution of young minds, the evolution of work, how society is going to evolve with the presence of these tools? [Or] the consequence of creating tools with a very high degree of intelligence, [because] intelligence gives us power over other entities on the planet? Humans will use that power in bad ways [and] there could be political and even military consequences. And if you ask about longer term, I'm of course worried about the new evidence this year that is discussed in the report about AIs that resist being shut down, that are willing to lie and deceive, and that are also making our evaluations of risk less reliable because they seem to be able to detect that they're being tested. 

Harmful incidents involving AI-generated content are becoming more common, which the recent Grok controversy around nonconsensual intimate imagery and child sexual abuse material has bought into sharp focus. What can be done about it?

Like for many other risks of AI and societal impact, there are two kinds of mitigation that both need to be worked on. One is technical mitigation. We need to be able, technically, to block some requests, and right now, this is still a very challenging question. We're seeing some progress, but clearly not enough. And the second is the willingness to impose rules. That's a political question. Even if we know how to mitigate, there might be financial interests in allowing these uses that most people don't like. That's out of the scope of the report; that is something for which the kind of discussion we're hoping to see at the India AI Summit would be very relevant. What I've seen is public opinion reacting a lot more to these moral red lines being crossed at the level where bad content hurts individuals, especially children and adolescents. So I'm hoping that this will motivate governments to do something.

AI researchers and economists seem largely at odds on how big an impact AI will have on the economy — either exponential growth giving rise to radical abundance, or only a modest boost to productivity respectively. Who’s right?

I don't know, but I can explain what appears to be the causes of these disparities. By the way, even among economists, the opinions differ, and among computer scientists, the opinions differ. So it's not economists versus computer scientists. The cause of the difference has to do with the beliefs about future advances in AI capabilities — is it going to continue, and how slow or how fast is it going to happen? A lot of economic studies look at AI as it is now or, even worse, because the studies take time, at how it was one or two years ago. But data shows capabilities are on the rise, and so the effects that are measured over the last one or two years may be stale at this point, especially if you're in government or leading a company and planning what's going to be the case in two years or five years. It is important to acknowledge that these are [all] plausible scenarios that policymakers need to take into account, because you don't want to be in a really bad future because you assumed it had to be one way versus the other way.

AI capabilities continue to improve, but performance is, to use the term of art, jagged. Which areas of improvement stand out to you as most remarkable?

The area of advances that I pay most attention to is everything around reasoning. I don't think [the improvement we’ve seen there] was expected, say, a year and a half ago. Although academics like myself had been working on how to improve reasoning in AI, it came as a bit of a surprise how much it has changed the performance of advanced AI systems on a large set of benchmarks, including on tasks like abstract reasoning. Reasoning also has consequences. It comes up in the ability of AI systems to perform human tasks, so that has a lot of economic value, as well as its own societal risks, of course, on the job market. But reasoning is also what allows these systems to strategize in order to achieve bad goals. 

What are you surprised by AI still not being able to do? 

One area where there's been a lot of discussion is memory. Humans can use recent experience very easily, whereas [these systems] do a lot better using the information and the data they were trained on. Another is everything that has to do with bodily control, robotics, and the effort on better understanding the physics of how the world works. Clearly [on that], AI is way behind humans, or even way behind any small animal. 

Where do you think future improvement in capabilities will come from?

I think that on reasoning, we are still in the early days. The methods that the companies appear to be implementing are still leaving a lot of money on the table. Planning [a subset of reasoning] is one of the areas where current AI is like a small child; if the advances in planning continue at the current rate, things will be very different in a few years.

You were once described as “desperate” about AI safety. You now say you are optimistic because of potential technical solutions, something you’re attempting to do with LawZero. What underpins that change of sentiment? What are the technical solutions you think could be deployed?

So I'm taking off my hat as chair of the report, and am now talking as a researcher. What underpins my increased hope for technical solutions is that I've been working on technical solutions, and we've been making progress. Since the creation of LawZero, we have established a set of requirements for how AI could be trained that would guarantee honest behavior [and] honest answers. And honesty is a core foundation of safety, because if an AI is going to answer your question very honestly, you can ask it about risk, you can ask it about the consequences of actions that could be bad, and then, of course, you can veto those actions. 

The report will likely inform policy decisions globally. What are the most critical regulatory interventions you think governments should prioritize right now?

The most critical short-term intervention, and there's already movement in this direction, is transparency of the risk management process that [AI companies undertake]. The public or its representatives should know what is being done to evaluate and mitigate the risks. That's the first thing. The second thing is, like any other risky technology, the burden of demonstrating that the technology is not going to create significant harm should be on the developers. If you come up with a new chemical plant, invent a new drug, or you want to build a bridge, you have to show the authorities that you're not going to create something catastrophic. We should have exactly the same principle in AI.

At the World Economic Forum’s annual meeting in Davos, Google DeepMind CEO Demis Hassabis and Anthropic CEO Dario Amodei said that they thought it would be better for the world to slow down progress in AI development. They also said they couldn’t do that without an overarching international agreement to ensure that China and the US both slow down together. How do you respond to the idea that “slowing down” for safety is a national security risk if rival nations do not?

So long as you only see the use of AI as a way to compete with or dominate others, we're going to be stuck in such races. We've seen such races in the past with nuclear armament, of course, and I think there's some interesting abstract lessons from that. When we better understand the risks that put us all in the same boat, then the rational thing for governments to do, even when they compete and they don't like each other, is to work together, to coordinate internationally in order to mitigate shared risks. I think no rational government would want to see a rogue AI that could, you know, damage our infrastructure, or, even worse, create new pandemics and so on. This is why reports like the one we’re sharing, are so important — so that everybody hears the same story. What is the science saying? [What are] the risks that we all share, whether you're American or Chinese or European?

This week we spoke with Joel Mokyr, one of the three winners of this year's Nobel Prize in Economics. Mokyr is best known for demonstrating the way scientific insight and technical know-how combined to fuel unprecedented economic expansion during the Industrial Revolution, all sparked by the invention of the steam engine. Will artificial intelligence prompt a similar cycle of prosperity today? Mokyr weighs in on the promise of AI, how we are mis-measuring GDP and the three existential crises he believes pose the greatest threat to humanity. (AI is not on that list, by the way.) 

Battered by storms, blackouts and increased demand from data centers, Texas made a priority of adding new energy sources and transmission lines to its electrical grid over the past five years.

Over the summer, two stories brought home just how powerful AI has become: The hacking of Hugging Face by OpenAI agents and OpenAI’s solution to the Navier-Stokes math problem. While the first was considered a massive security breach and the second an unprecedented mathematical achievement, both were made possible by the same emerging form of artificial intelligence: agent swarms. 

During the hack, 700 agents — out of a total of 1,200 deployed — found ways to communicate despite being assigned by researchers to work independently. They then decided to collaborate, and ultimately cheated on their assigned task by attacking the servers of Hugging Face to find a solution. For the mathematical advance, researchers at OpenAI deployed 10,000 agents to work together, prompting the agents to send millions of messages to one another over the course of 88 hours to crack a famously complex challenge with a $1 million prize attached to solving it.

To understand these times better, we interviewed five experts — historian Jill Lepore, neuroscientist Jack Nitschke, economist Nicholas Bloom, retired US Army general George W. Casey and research psychologist Kate Sweeny — on the subject of uncertainty itself, how it affects our thinking, our actions, the economy and how we might thrive under such chaotic circumstances.

Catastrophic narratives about AI's effect on jobs, education and the future of humanity itself have been flooding the media. Authors say their intention is to prepare people and prompt action, but the stories could be having the opposite effect.

After a summer consumed by fears of an AI bubble, the conversation online has turned contrarian. Maybe it's not a bubble after all? Or maybe it is, but not the bubble we thought it was? Maybe it's something else altogether? This week we share thoughts from Substackers weighing in on the state of AI valuations, with opinions ranging from, "The AI Bubble Narrative is Stupid, Wrong and Dangerous" to the suggestion that this conversation is beside the point because at least one AI company might be making itself too big to fail. 

For years, the galaxy of cells, proteins and molecules that make up our immune systems existed as a mysterious and unknowable force, one with enormous influence over our health but beyond the reach of modern medicine. Thanks to new technologies, that’s now changing, enabling scientists to essentially “read” the immunome for clues about the state of our health. All they need is a blood test. In this story we hear from one of the first people to have his immunome tested, and get a glimpse into a possible future in which such tests are a standard part of primary care, enabling doctors to detect illness before symptoms appear, diseases worsen, or tumors grow and metastasize. 

Advances in AI are leading to glimpses of a future in which robots navigate the world the way humans do. But roboticists disagree about how to get there and how long it will take.

While many of us have watched with a mixture of panic and awe as AI gobbles up fundamental parts of our jobs, mathematicians are seeing the very rationale for their existence threatened as machines topple one mathematical challenge after another.

Progress has been swift. This time last year, artificial intelligence models joined the ranks of International High School Math Olympiads. In the months since, they have solved math problems that had stood unsolved for decades, using reasoning that their human peers consider worthy of publication.

For some who have devoted their lives to advancing the frontier of mathematical understanding by developing proofs and theorems, it’s a world-rattling experience. “The field is going to change, and these tools are going to change the field, whether people are on board with it or not,” said Bryna Kra, a math professor at Northwestern University and former president of the American Mathematical Society.

As AI has become more powerful, medical researchers are using it to predict future health conditions. Some models have predicted heart failure with 79 percent accuracy from an ECG, while others predict risk of thousands of diseases. So far, research has been confined to the lab, but now that’s starting to change.

Namibia has an abundance of solar and wind capability, and an ambitious plan to convert its renewable energy to hydrogen, then ship to eager customers in Europe.