If anyone builds it, everyone dies why superhuman AI would kill us all

"In 2023, hundreds of AI luminaries signed an open letter warning that artificial intelligence poses a serious risk of human extinction. Since then, the AI race has only intensified. Companies and countries are rushing to build machines that will be smarter than any person. And the world is devastat...

Full description

Saved in:
Bibliographic Details
Main Authors: Yudkowsky, Eliezer, 1979- (Author), Soares, Nate (Author)
Format: Books Print Book
Language:English
Published: New York : Little, Brown and Company, 2025.
Edition:First edition.
Subjects:
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:"In 2023, hundreds of AI luminaries signed an open letter warning that artificial intelligence poses a serious risk of human extinction. Since then, the AI race has only intensified. Companies and countries are rushing to build machines that will be smarter than any person. And the world is devastatingly unprepared for what would come next. For decades, two signatories of that letter--Eliezer Yudkowsky and Nate Soares--have studied how smarter-than-human intelligences will think, behave, and pursue their objectives. Their research says that sufficiently smart AIs will develop goals of their own that put them in conflict with us--and that if it comes to conflict, an artificial superintelligence would crush us. The contest wouldn't even be close. How could a machine superintelligence wipe out our entire species? Why would it want to? Would it want anything at all? In this urgent book, Yudkowsky and Soares walk through the theory and the evidence, present one possible extinction scenario, and explain what it would take for humanity to survive. The world is racing to build something truly new under the sun. And if anyone builds it, everyone dies"--
Item Description:Includes QR codes with URLs at the end of each chapter linking to a supplemental website (IfAnyoneBuildsIt.com) with more information expanding on that chapter.
Physical Description:xii, 259 pages ; 25 cm
Bibliography:Includes bibliographical references (pages 236-252) and index.
ISBN:9780316595643
0316595640
9780316601115
031660111X
Author Notes:

ELIEZER YUDKOWSKY is one of the founding researchers of the field of AGI alignment, which is concerned with understanding how smarter-than-human intelligences think, behave, and pursue their goals. He appeared on TIME magazine's list of the 100 Most Influential People In AI, was one of the twelve public figures featured in The New York Times' s "Who's Who Behind the Dawn of the Modern Artificial Intelligence Movement," and was one of the seven thought leaders spotlighted in The Washington Post' s discussion of "AI's Rival Factions." He spoke on the main stage at 2023's TED conference and has been discussed or interviewed in The New Yorker, Newsweek, Forbes, Wired, Bloomberg, The Atlantic, The Economist, and many other venues. He has close to 200,000 followers on X, where he frequently dialogues with prominent public figures including the heads of frontier AI labs.



NATE SOARES is the President of MIRI. He has been working in the field for over a decade, after previous experience at Microsoft and Google. Soares is the author of a large body of technical and semi-technical writing on AI alignment, has been interviewed in Vanity Fair and the Financial Times, and has spoken on conference panels alongside many of the AI field's leaders.