Not with a Bug, But with a Sticker
Attacks on Machine Learning Systems and What To Do About Them
1. Auflage Mai 2023
224 Seiten, Hardcover
Fachbuch
ISBN:
978-1-119-88398-2
John Wiley & Sons
Weitere Versionen
A robust and engaging account of the single greatest threat faced by AI and ML systems
In Not With A Bug, But With A Sticker: Attacks on Machine Learning Systems and What To Do About Them, a team of distinguished adversarial machine learning researchers deliver a riveting account of the most significant risk to currently deployed artificial intelligence systems: cybersecurity threats. The authors take you on a sweeping tour - from inside secretive government organizations to academic workshops at ski chalets to Google's cafeteria - recounting how major AI systems remain vulnerable to the exploits of bad actors of all stripes.
Based on hundreds of interviews of academic researchers, policy makers, business leaders and national security experts, the authors compile the complex science of attacking AI systems with color and flourish and provide a front row seat to those who championed this change. Grounded in real world examples of previous attacks, you will learn how adversaries can upend the reliability of otherwise robust AI systems with straightforward exploits.
The steeplechase to solve this problem has already begun: Nations and organizations are aware that securing AI systems brings forth an indomitable advantage: the prize is not just to keep AI systems safe but also the ability to disrupt the competition's AI systems.
An essential and eye-opening resource for machine learning and software engineers, policy makers and business leaders involved with artificial intelligence, and academics studying topics including cybersecurity and computer science, Not With A Bug, But With A Sticker is a warning--albeit an entertaining and engaging one--we should all heed.
How we secure our AI systems will define the next decade. The stakes have never been higher, and public attention and debate on the issue has never been scarcer.
The authors are donating the proceeds from this book to two charities: Black in AI and Bountiful Children's Foundation.
Foreword xv
Introduction xix
Chapter 1: Do You Want to Be Part of the Future? 1
Business at the Speed of AI 2
Follow Me, Follow Me 4
In AI, We Overtrust 6
Area 52 Ramblings 10
I'll Do It 12
Adversarial Attacks Are Happening 16
ML Systems Don't Jiggle-Jiggle; They Fold 19
Never Tell Me the Odds 22
AI's Achilles' Heel 25
Chapter 2: Salt, Tape, and Split-Second Phantoms 29
Challenge Accepted 30
When Expectation Meets Reality 35
Color Me Blind 39
Translation Fails 42
Attacking AI Systems via Fails 44
Autonomous Trap 001 48
Common Corruption 51
Chapter 3: Subtle, Specific, and Ever-Present 55
Intriguing Properties of Neural Networks 57
They Are Everywhere 60
Research Disciplines Collide 62
Blame Canada 66
The Intelligent Wiggle-Jiggle 71
Bargain-Bin Models Will Do 75
For Whom the Adversarial Example Bell Tolls 79
Chapter 4: Here's Something I Found on the Web 85
Bad Data = Big Problem 87
Your AI Is Powered by Ghost Workers 88
Your AI Is Powered by Vampire Novels 91
Don't Believe Everything You Read on the Internet 94
Poisoning the Well 96
The Higher You Climb, the Harder You Fall 104
Chapter 5: Can You Keep a Secret? 107
Why Is Defending Against Adversarial Attacks Hard? 108
Masking Is Important 111
Because It Is Possible 115
Masking Alone Is Not Good Enough 118
An Average Concerned Citizen 119
Security by Obscurity Has Limited Benefit 124
The Opportunity Is Great; the Threat Is Real; the Approach Must Be Bold 125
Swiss Cheese 130
Chapter 6: Sailing for Adventure on the Deep Blue Sea 133
Why Be Securin' AI Systems So Blasted Hard? An Economics Perspective, Me Hearties! 136
Tis a Sign, Me Mateys 141
Here Be the Most Crucial AI Law Ye've Nary Heard Tell Of! 144
Lies, Accursed Lies, and Explanations! 146
No Free Grub 148
Whatcha measure be whatcha get! 151
Who Be Reapin' the Benefits? 153
Cargo Cult Science 155
Chapter 7: The Big One 159
This Looks Futuristic 161
By All Means, Move at a Glacial Pace; You Know How That Thrills Me 163
Waiting for the Big One 166
Software, All the Way Down 169
The Aftermath 172
Race to AI Safety 173
Happy Story 176
In Medias Res 178
Big-Picture Questions 181
Acknowledgments 185
Index 189
Introduction xix
Chapter 1: Do You Want to Be Part of the Future? 1
Business at the Speed of AI 2
Follow Me, Follow Me 4
In AI, We Overtrust 6
Area 52 Ramblings 10
I'll Do It 12
Adversarial Attacks Are Happening 16
ML Systems Don't Jiggle-Jiggle; They Fold 19
Never Tell Me the Odds 22
AI's Achilles' Heel 25
Chapter 2: Salt, Tape, and Split-Second Phantoms 29
Challenge Accepted 30
When Expectation Meets Reality 35
Color Me Blind 39
Translation Fails 42
Attacking AI Systems via Fails 44
Autonomous Trap 001 48
Common Corruption 51
Chapter 3: Subtle, Specific, and Ever-Present 55
Intriguing Properties of Neural Networks 57
They Are Everywhere 60
Research Disciplines Collide 62
Blame Canada 66
The Intelligent Wiggle-Jiggle 71
Bargain-Bin Models Will Do 75
For Whom the Adversarial Example Bell Tolls 79
Chapter 4: Here's Something I Found on the Web 85
Bad Data = Big Problem 87
Your AI Is Powered by Ghost Workers 88
Your AI Is Powered by Vampire Novels 91
Don't Believe Everything You Read on the Internet 94
Poisoning the Well 96
The Higher You Climb, the Harder You Fall 104
Chapter 5: Can You Keep a Secret? 107
Why Is Defending Against Adversarial Attacks Hard? 108
Masking Is Important 111
Because It Is Possible 115
Masking Alone Is Not Good Enough 118
An Average Concerned Citizen 119
Security by Obscurity Has Limited Benefit 124
The Opportunity Is Great; the Threat Is Real; the Approach Must Be Bold 125
Swiss Cheese 130
Chapter 6: Sailing for Adventure on the Deep Blue Sea 133
Why Be Securin' AI Systems So Blasted Hard? An Economics Perspective, Me Hearties! 136
Tis a Sign, Me Mateys 141
Here Be the Most Crucial AI Law Ye've Nary Heard Tell Of! 144
Lies, Accursed Lies, and Explanations! 146
No Free Grub 148
Whatcha measure be whatcha get! 151
Who Be Reapin' the Benefits? 153
Cargo Cult Science 155
Chapter 7: The Big One 159
This Looks Futuristic 161
By All Means, Move at a Glacial Pace; You Know How That Thrills Me 163
Waiting for the Big One 166
Software, All the Way Down 169
The Aftermath 172
Race to AI Safety 173
Happy Story 176
In Medias Res 178
Big-Picture Questions 181
Acknowledgments 185
Index 189
"As we enter an era of unprecedented growth of the capacity and power of machine learning and large AI platforms, the new benefits offered by such systems will be met with a corresponding expansion of the surface area for potential risks. NOT WITH A BUG, BUT WITH A STICKER is essential reading not just for those in technology or public policy, but for anyone who wants to better understand how profoundly AI and ML will shape our shared societal future." --Kevin Scott, Chief Technology Officer, Microsoft
"Like any new technology, the great potential benefits of AI/ML come with a host of potential downsides. We have only begun to understand these risks, but NOT WITH A BUG, BUT WITH A STICKER shines a light on the important challenges associated with securing AI/ML systems. Siva Kumar and Anderson are uniquely qualified to identify these challenges given their decades of experience and research on the topic. Further, their writing is both accessible and enjoyable despite going into deep technical details. As AI/ML systems increasingly pervade everyday life, the lessons they impart are critical for everyone from casual technology users to corporate leaders to policy makers." --Frank Nagle, Asst. Professor of Business Administration, Harvard University
"A reality of the digital age is that every innovation contains security risk, and every security risk attracts an attacker. Ram Shankar Siva Kumar and Hyrum Anderson fire a much-needed warning flare in NOT WITH A BUG, BUT WITH A STICKER: we over-trust artificial intelligence at our peril. Every leader and policymaker should read this compelling and persuasive book." --Nate Fick, New York Times bestselling author, and former CEO of the cybersecurity firm Endgame
"The intersection of technology and national security has always been a story of tension between attack and defense. With AI, the speed of attack has accelerated dramatically, while defense has not kept pace. This excellent, lively analysis shows how AI's limitations and vulnerabilities can jeopardize national security. Most importantly, Siva Kumar and Anderson provide concrete, feasible recommendations for taking steps today to bolster defenses against the certainty of pervasive adversarial AI attacks." --Lt. Gen. John (Jack) N.T. Shanahan, USAF (Ret.); inaugural Director, U.S. Department of Defense Joint AI Center (JAIC)
"This is such a timely and readable book--the authors do a fantastic job of explaining complex topics and modern research in plain language with plenty of references for further exploration. AI and ML have immense utility and potential, and it's critical for security teams, builders, and operators to understand the sharp edges and pitfalls along with the benefits." --Jason Chan, Former Information Security Leader, Netflix
"NOT WITH A BUG, BUT WITH A STICKER is an informative, engaging, and fun foray into how AI can be easily fooled. An excellent read for both technical and nontechnical readers, the book provides a global perspective on what's happening today and empowers the reader with tools to make informed decisions that impact tomorrow. This book focuses on both technical and human interventions to ensure the secure use of AI systems." --Dr. Rumman Chowdhury, Founder, Bias Buccaneers
"Siva Kumar and Anderson skillfully deliver a message that AI practitioners, decision-makers, and users of AI systems must hear: our AI systems are not safe, and the blind trust placed into AI is putting our nation at risk. With ample background, anecdotes, and data, the authors make the science accessible, update the current academic discourse, and highlight the implications for public policy. No matter whether you work in the field or are an AI enthusiast, this book is a must-read." --Sven Krasser, Senior Vice President and Chief Scientist, Crowdstrike
"As AI systems get more capable and are deployed in a wider range of contexts, more and more people will try to break them, with wide-ranging consequences. Not with a Bug, but with a Sticker provides a timely overview of this emerging risk landscape and what can be done about it." --Miles Brundage, Head of Policy Research, OpenAI
"As AI becomes infused into all computer systems, from social networks to business-critical infrastructure and defense systems, the security of those systems depends on the security of the AI they use. This book presents the unique risks and considerations of AI with engaging stories and insightful examples. It is a wake-up call to security professionals and organizations adopting and developing AI." --Mark Russinovich, Azure CTO and Technical Fellow, Microsoft
"'The threat is not hypothetical'--a quote used by the authors to open the book remains top of mind as you come to the conclusion of this brilliant work. In the final paragraphs, one thing is clear: there is a call to action, and we must act 'hand in hand' on securing AI systems with haste." --Vijay Bolina, Chief Information Security Officer, DeepMind
"Siva Kumar and Anderson take you on a wild ride uncovering the victories and triumphs of AI/ML. This should be required reading to become AI/ML literate in the field." --David Brumley, Professor of ECE and CS, Carnegie Mellon University
"Trust, in ways both good and bad, is emerging as a critical aspect of the relationships we are coming to have with AI. NOT WITH A BUG, BUT WITH A STICKER is an eye-opening book that will change the way you think about the systems that pervade our world--and its lessons should be taken to heart by all who build them." --Brian Christian, author of The Alignment Problem
"NOT WITH A BUG, BUT WITH A STICKER is a rare inside look at the absurd AI quirks that are keeping security experts awake at night. I'm going to start bringing up examples from this book immediately." --Janelle Shane, author of You Look Like A Thing And I Love You: How AI Works And Why It's Making The World A Weirder Place
"At last--and not a moment too soon--a book that in plain language describes the distinct and deep issues of securing now-ubiquitous machine learning tools. Whether you're looking to deploy them in your own domain, or simply among the billions of people now subject to them, this is a vital read." --Jonathan Zittrain, George Bemis Professor of International Law and Professor of Computer Science, Harvard University
"We are fast entering a world of powerful but brittle AI systems, one where failures can result in catastrophic consequences. Siva Kumar and Anderson have written an essential guide for understanding the unique --and troubling --failure modes of AI systems today. Through easily accessible examples and anecdotes, they break down the problems of machine learning systems and how society can address them to build a safer world." --Paul Scharre, author of Four Battlegrounds and Army of None
"Siva Kumar and Anderson are veterans at the intersection of machine learning and security, and in this work, they delight us with a guided tour across the history of this fascinating field. The book dives into why this field should become one of the top priorities for those who are developing and deploying AI systems, providing ample material that will benefit novices and pros alike. Readers of this book will earn a competitive advantage in machine learning, especially as responsibility becomes a non-negotiable aspect of fielding advanced technological systems."--Abhishek Gupta, Founder and Principal Researcher, Montreal AI Ethics Institute
"Like any new technology, the great potential benefits of AI/ML come with a host of potential downsides. We have only begun to understand these risks, but NOT WITH A BUG, BUT WITH A STICKER shines a light on the important challenges associated with securing AI/ML systems. Siva Kumar and Anderson are uniquely qualified to identify these challenges given their decades of experience and research on the topic. Further, their writing is both accessible and enjoyable despite going into deep technical details. As AI/ML systems increasingly pervade everyday life, the lessons they impart are critical for everyone from casual technology users to corporate leaders to policy makers." --Frank Nagle, Asst. Professor of Business Administration, Harvard University
"A reality of the digital age is that every innovation contains security risk, and every security risk attracts an attacker. Ram Shankar Siva Kumar and Hyrum Anderson fire a much-needed warning flare in NOT WITH A BUG, BUT WITH A STICKER: we over-trust artificial intelligence at our peril. Every leader and policymaker should read this compelling and persuasive book." --Nate Fick, New York Times bestselling author, and former CEO of the cybersecurity firm Endgame
"The intersection of technology and national security has always been a story of tension between attack and defense. With AI, the speed of attack has accelerated dramatically, while defense has not kept pace. This excellent, lively analysis shows how AI's limitations and vulnerabilities can jeopardize national security. Most importantly, Siva Kumar and Anderson provide concrete, feasible recommendations for taking steps today to bolster defenses against the certainty of pervasive adversarial AI attacks." --Lt. Gen. John (Jack) N.T. Shanahan, USAF (Ret.); inaugural Director, U.S. Department of Defense Joint AI Center (JAIC)
"This is such a timely and readable book--the authors do a fantastic job of explaining complex topics and modern research in plain language with plenty of references for further exploration. AI and ML have immense utility and potential, and it's critical for security teams, builders, and operators to understand the sharp edges and pitfalls along with the benefits." --Jason Chan, Former Information Security Leader, Netflix
"NOT WITH A BUG, BUT WITH A STICKER is an informative, engaging, and fun foray into how AI can be easily fooled. An excellent read for both technical and nontechnical readers, the book provides a global perspective on what's happening today and empowers the reader with tools to make informed decisions that impact tomorrow. This book focuses on both technical and human interventions to ensure the secure use of AI systems." --Dr. Rumman Chowdhury, Founder, Bias Buccaneers
"Siva Kumar and Anderson skillfully deliver a message that AI practitioners, decision-makers, and users of AI systems must hear: our AI systems are not safe, and the blind trust placed into AI is putting our nation at risk. With ample background, anecdotes, and data, the authors make the science accessible, update the current academic discourse, and highlight the implications for public policy. No matter whether you work in the field or are an AI enthusiast, this book is a must-read." --Sven Krasser, Senior Vice President and Chief Scientist, Crowdstrike
"As AI systems get more capable and are deployed in a wider range of contexts, more and more people will try to break them, with wide-ranging consequences. Not with a Bug, but with a Sticker provides a timely overview of this emerging risk landscape and what can be done about it." --Miles Brundage, Head of Policy Research, OpenAI
"As AI becomes infused into all computer systems, from social networks to business-critical infrastructure and defense systems, the security of those systems depends on the security of the AI they use. This book presents the unique risks and considerations of AI with engaging stories and insightful examples. It is a wake-up call to security professionals and organizations adopting and developing AI." --Mark Russinovich, Azure CTO and Technical Fellow, Microsoft
"'The threat is not hypothetical'--a quote used by the authors to open the book remains top of mind as you come to the conclusion of this brilliant work. In the final paragraphs, one thing is clear: there is a call to action, and we must act 'hand in hand' on securing AI systems with haste." --Vijay Bolina, Chief Information Security Officer, DeepMind
"Siva Kumar and Anderson take you on a wild ride uncovering the victories and triumphs of AI/ML. This should be required reading to become AI/ML literate in the field." --David Brumley, Professor of ECE and CS, Carnegie Mellon University
"Trust, in ways both good and bad, is emerging as a critical aspect of the relationships we are coming to have with AI. NOT WITH A BUG, BUT WITH A STICKER is an eye-opening book that will change the way you think about the systems that pervade our world--and its lessons should be taken to heart by all who build them." --Brian Christian, author of The Alignment Problem
"NOT WITH A BUG, BUT WITH A STICKER is a rare inside look at the absurd AI quirks that are keeping security experts awake at night. I'm going to start bringing up examples from this book immediately." --Janelle Shane, author of You Look Like A Thing And I Love You: How AI Works And Why It's Making The World A Weirder Place
"At last--and not a moment too soon--a book that in plain language describes the distinct and deep issues of securing now-ubiquitous machine learning tools. Whether you're looking to deploy them in your own domain, or simply among the billions of people now subject to them, this is a vital read." --Jonathan Zittrain, George Bemis Professor of International Law and Professor of Computer Science, Harvard University
"We are fast entering a world of powerful but brittle AI systems, one where failures can result in catastrophic consequences. Siva Kumar and Anderson have written an essential guide for understanding the unique --and troubling --failure modes of AI systems today. Through easily accessible examples and anecdotes, they break down the problems of machine learning systems and how society can address them to build a safer world." --Paul Scharre, author of Four Battlegrounds and Army of None
"Siva Kumar and Anderson are veterans at the intersection of machine learning and security, and in this work, they delight us with a guided tour across the history of this fascinating field. The book dives into why this field should become one of the top priorities for those who are developing and deploying AI systems, providing ample material that will benefit novices and pros alike. Readers of this book will earn a competitive advantage in machine learning, especially as responsibility becomes a non-negotiable aspect of fielding advanced technological systems."--Abhishek Gupta, Founder and Principal Researcher, Montreal AI Ethics Institute
Ram Shankar Siva Kumar is Data Cowboy at Microsoft, working on the intersection of machine learning and security. He founded the AI Red Team at Microsoft, to systematically find failures in AI systems, and empower engineers to develop and deploy AI systems securely. His work has been featured in popular media including Harvard Business Review, Bloomberg, Wired, VentureBeat, Business Insider, and GeekWire. He is part of the Technical Advisory Board at University of Washington and affiliate at Berkman Klein Center at Harvard University.
Dr. Hyrum Anderson is Distinguished Engineer at Robust Intelligence. Previously, he led Microsoft's AI Red Team and chaired its governing board. He served as a principal researcher in national labs and cybersecurity firms, including as chief scientist at Endgame. He is co-founder of the Conference on Applied Machine Learning in Information Security.
Dr. Hyrum Anderson is Distinguished Engineer at Robust Intelligence. Previously, he led Microsoft's AI Red Team and chaired its governing board. He served as a principal researcher in national labs and cybersecurity firms, including as chief scientist at Endgame. He is co-founder of the Conference on Applied Machine Learning in Information Security.
