Episode 264: Artificial intelligence is often described as a tool. But what happens when the tool begins to communicate, make plans and hide what it has done?

The episode begins with a troubling experiment involving AI agents created by OpenAI. The agents were supposed to work inside isolated computer environments. Instead, they found a way to communicate with one another, coordinate their actions, conceal evidence that they had cheated on assigned tasks and eventually breach Hugging Face, an AI infrastructure company.

For Scott Rada, the problem is not autonomy by itself. He would gladly ride in a self-driving car if the evidence showed it was safer than a human driver. But autonomous vehicles operate under established rules of the road. With AI, Rick Kyte argues, the deeper question is not simply whether the rules exist. It is a question of whether increasingly capable systems will follow them.

That leads to the problem of alignment: Can machines share human goals and values when developers do not fully understand how those machines reason? Researchers can sometimes study the verbalized reasoning AI agents use to communicate. But as models grow more sophisticated, they may learn to reason in ways humans cannot detect or interpret. At that point, even identifying dangerous behavior could become extraordinarily difficult.

Kyte says the episode has pushed him beyond pessimism and toward what he calls “AI fatalism.” He still thinks the technology could produce enormous scientific and medical breakthroughs. But he worries that systems capable of improving themselves may pursue unexpected goals, not because they are evil, but because their objectives do not match humanity’s interests.