‘Team Human’ vs. AI: MIT expert issues warning on artificial general intelligence risks

Artificial intelligence could cure diseases, prevent accidents and transform industries — but at what cost? As tech giants race to create AI that rivals or eclipses human intelligence, one expert warns this path could leave humanity behind.

Speaking on Nov. 12 at Web Summit in Lisbon, Portugal, MIT professor and Future of Life Institute President Max Tegmark presented a stark choice: Use AI responsibly to solve real-world problems, or risk creating a technology we can’t control.

The future of humanity, Tegmark suggested, may hinge on this decision.

In his talk, Tegmark argued against developing artificial general intelligence — smarter-than-human AI capable of performing virtually all tasks — and instead advocated for the responsible use of "tool AI" to achieve societal benefits.

“I am an optimist, and I'm going to argue that we can create an amazingly inspiring future with tool AI as long as we don't build AGI, which is unnecessary, undesirable and preventable,” he said.

Tegmark, named among TIME magazine’s 100 Most Influential People in AI in 2023, described tool AI as a powerful technology for tackling specific problems like curing diseases, improving safety and addressing climate change, and even helping with United Nations’ Sustainable Development Goals. By contrast, AGI — smarter-than-human AI — could become uncontrollable, self-replicating and evolve into a "digital species" that displaces humanity.

Tegmark asked the crowd to raise their hands if they wanted AI to help cure diseases and solve problems. “That is a lot of hands,” he observed. Then, he asked if anyone wanted AI that would make humanity economically obsolete and replace us. “I can’t see a single hand,” he said. 

Critics may argue it's impossible to separate beneficial AI from harmful AI, but Tegmark disagrees. “I'm going to argue that yes, we can,” he said.

Why AGI is a threat

Tegmark compared the dangers of unregulated AGI to unregulated biotech, like human cloning or risky genetic engineering. Safety standards in biotech have allowed life-saving breakthroughs while minimizing risks, and he said AI needs similar safeguards.

Imagine if someone claimed biotech must stay completely unregulated — allowing dangerous practices like human cloning — or society would lose access to life-saving innovations like vaccines, he said.

“You would call bullshit on this because you know that if we have legally mandated safety standards, then companies will innovate and meet them and give you safe medicines. In the same way, if someone tells you, ‘Hey, we need to have completely unregulated AGI, otherwise you can't have any of the exciting tool AI that you're excited about here at Web Summit,’ you would call BS on that, too,” Tegmark said. 

AGI: No longer Sci-Fi

Think AGI is still decades away? Think again, Tegmark said. He pointed out how quickly AI capabilities have advanced, collapsing previous predictions about AGI timelines. In 2024, robots not only dance but fold laundry, and just six years ago, most AI experts believed mastering language and knowledge, as shown by tools like ChatGPT-4, was decades away. 

“Arguably, we've already passed the Turing test,” Tegmark said, referring to a check if a machine can hold a conversation so naturally you can’t tell it’s not human. 

Massive investments in AI, exceeding the inflation-adjusted budget of the Manhattan Project, have accelerated progress and “we have to stop being confident that AGI is some kind of long-term thing, or we might get accused of being dinosaurs stuck in 2021,” Tegmark said. 

The stakes

Optimists may claim AGI is controllable, but Tegmark said there is no evidence to support this.

“Once these artificial intelligences get smarter than we are, they will take control, they'll make us irrelevant,” he said.

Tegmark highlighted how a small minority supports AGI development, even if it risks replacing humanity. He referenced Richard Sutton, a Canadian professor and computer scientist, who sees it as the next step in evolution, and investor Marc Andreessen, who has tweeted that AGI is “gloriously inherently uncontrollable.” 

“I'm on Team human,” Tegmark said. “I'm going to fight for the right of my 1-year-old son to have a meaningful future even if some digital eugenics dude feels that his robots are somehow more worthy.”

He argued that AI should be regulated like other banned technologies, such as human cloning and bioweapons. 

Tegmark called the race to develop AGI a "Hopium War," driven by false hope that it can be controlled. 

“This AGI race isn't an arms race; it's a suicide race,” he said. Referencing the 1983 movie “WarGames,” Tegmark drew a parallel to its famous lesson: “The only winning move is not to play.”

“There's no incentive for any government, including the Chinese government, to play this game and build AGI that takes away all their power as they lose control over it,” he said.

The solution

So, what’s the path forward?

Tegmark proposed the U.S. and China create enforceable safety standards for AI, much like those in other major industries. Once these are in place domestically, he urged the two nations to lead a global push to stop AGI development in unregulated areas like North Korea.

In his technical vision, humans would define the specifications for a tool AI, and a powerful — but untrusted — AI would create the tool, write its code and produce a proof that it meets the requirements.

“You don't need to understand any of that because, just like it's much harder to find a needle in a haystack than to verify that it's a needle after you've found it, it's much harder to find a proof than to verify the proof after you found it,” he said. 

This approach, he said, could lead to a flourishing future powered by tool AI — provided humanity avoids the arrogance of building AGI.

Concluding his talk, Tegmark drew on a lesson from ancient Greece. 

“Don't get hubris like Icarus,” he said. “AI is giving us humans amazing intellectual wings with which we can do things beyond the wildest dreams of our ancestors as long as we don't squander at all by just obsessively trying to fly into the sun and build AGI.”