The Failure of Risk Management: Why It's Broken and How to Fix It »

Douglas W. Hubbard is the inventor of Applied Information Economics (AIE). He is an internationally recognized expert in the field of measuring intangibles, risks, and value, especially in IT value, and is a popular speaker at numerous conferences. He has written articles for InformationWeek, CIO Enterprise, and DBMS magazine. His AIE method has been applied to dozens of large Fortune 500 IT investments, military logistics, venture capital, aerospace, and environmental issues. Doug is the author of How to Measure Anything: Finding the Value of Intangibles in Business (Wiley).

The 2008 credit crisis, terrorism, Katrina, computer hackers, and air travel disasters all have something in common-the methods used to assess and manage these risks are fundamentally flawed. If risks cannot be properly evaluated, risk management itself becomes the biggest risk. The Failure of Risk Management shows you how to identify and fix these hidden problems in risk management.

Ineffective risk management methods, often touted as "best practices," are passed from company to company like a bad virus with a long incubation period: there are no early indicators of ill effects until it's too late and catastrophe strikes. Exploring why risk management fails—the failure to measure and validate methods as a whole or in part; the use of components known not to work; and not using components that are known to work—The Failure of Risk Management shows you how to measure the performance of risk management in a meaningful way, identify where risk management is broken, and fix it.

Respected expert and bestselling author Douglas Hubbard-creator of the critically praised Applied Information Economics (AIE)—uses real-world examples to reveal the serious problems in our current approaches to risk analysis. Hubbard skillfully illustrates how to use a calibrated risk analyses approach, and the many benefits that go along with it, along with checklists and practice examples to get you started.

One of the first resources to apply risk management across all industries, The Failure of Risk Management provides you with the tools you need to hit the ground running with radically better risk management solutions.

Here, you'll discover:

• The diversity of approaches to assess and mitigate risks

• Why many influential methods-both qualitative and quantitative don't work

• Why we shouldn't always trust assessments based on "experience" alone

• The fallacies that stop you from adopting better risk management methods

• How those who develop models of risks justify (in error) excluding the biggest risks

• Adding empirical science to risk management

Preface xi

Acknowledgments xv

Part 1 An Introduction to the Crisis 1

Chapter 1 Healthy Skepticism for Risk Management 3

Common Mode Failure 4

What Counts as Risk Management 8

Anecdote: The Risk of Outsourcing Drug Manufacturing 11

What Failure Means 16

Scope and Objectives of This Book 18

Chapter 2 Risk Management: A Very Short Introduction to Where We've Been and Where (We Think) We Are 21

The Entire History of Risk Management (in 800 Words or Less) 22

Methods of Assessing Risks 24

Risk Mitigation 26

The State of Risk Management According to Surveys 31

Chapter 3 How Do We Know What Works? 37

An Assessment of Self-Assessments 37

Potential Objective Evaluations of Risk Management 42

What We May Find 49

Part 2 Why It's Broken 53

Chapter 4 The "Four Horsemen" of Risk Management: Some (Mostly) Sincere Attempts to Prevent an Apocalypse 55

Actuaries 57

War Quants: How World War II Changed Risk Analysis Forever 59

Economists 63

Management Consulting: How a Power Tie and a Good Pitch Changed Risk Management 68

Comparing the Horsemen 74

Major Risk Management Problems to Be Addressed 76

Chapter 5 An Ivory Tower of Babel: Fixing the Confusion about Risk 79

The Frank Knight Definition 81

Risk as Volatility 84

A Construction Engineering Definition 86

Risk as Expected Loss 86

Risk as a Good Thing 88

Risk Analysis and Risk Management versus Decision Analysis 90

Enriching the Lexicon 91

Chapter 6 The Limits of Expert Knowledge: Why We Don't Know What We Think We Know about Uncertainty 95

The Right Stuff: How a Group of Psychologists Saved Risk Analysis 97

Mental Math: Why We Shouldn't Trust theNumbers in Our Heads 99

"Catastrophic" Overconfidence 102

The Mind of "Aces": Possible Causes and Consequences of Overconfidence 107

Inconsistencies and Artifacts: What Shouldn't Matter Does 111

Answers to Calibration Tests 115

Chapter 7 Worse Than Useless: The Most Popular Risk Assessment Method and Why It Doesn't Work 117

A Basic Course in Scoring Methods (Actually, It's the Advanced Course, Too-There's Not Much to Know) 118

Does That Come in "Medium"?: Why Ambiguity Does Not Offset Uncertainty 123

Unintended Effects of Scales: What You Don't Know Can Hurt You 130

Clarification of Scores and Preferences: Different but Similar-Sounding Methods and Similar but Different-Sounding Methods 135

Chapter 8 Black Swans, Red Herrings, and Invisible Dragons: Overcoming Conceptual Obstacles to Improved Risk Management 145

Risk and Righteous Indignation: The Belief that Quantitative Risk Analysis Is Impossible 146

A Note about Black Swans 151

Frequentist versus Subjectivist 158

We're Special: The Belief that Risk Analysis Might Work, But Not Here 161

Chapter 9 Where Even the Quants Go Wrong: Common and Fundamental Errors in Quantitative Models 167

Introduction to Monte Carlo Concepts 168

Survey of Monte Carlo Users 172

The Risk Paradox 174

The Measurement Inversion 176

Where's the Science? The Lack of Empiricism in Risk Models 178

Financial Models and the Shape of Disaster: Why Normal Isn't so Normal 181

Following Your Inner Cow: The Problem with Correlations 187

"That's Too Uncertain": How Modelers Justify Excluding the Biggest Risks 191

Is Monte Carlo Too Complicated? 195

Part 3 How to Fix It 199

Chapter 10 The Language of Uncertain Systems: The First Step Toward Improved Risk Management 201

Getting Your Probabilities Calibrated 203

The Model of Uncertainty: Decomposing Risk with Monte Carlos 208

Decomposing Probabilities: Thinking about Chance the Way You Think about a Budget 212

A Few Modeling Principles 213

Modeling the Mechanism 215

Chapter 11 The Outward-Looking Modeler: Adding Empirical Science to Risk 221

Why Your Model Won't Behave 223

Empirical Inputs 224

Introduction to Bayes: One Way to Get around that "Limited Data for Disasters" Problem 227

Self-Examinations for Modelers Who Care about Quality 233

Chapter 12 The Risk Community: Intra-and Extraorganizational Issues of Risk Management 241

Getting Organized 242

Managing the Global Probability Model 244

Incentives for a Calibrated Culture 250

Extraorganizational Issues: Solutions beyond Your Office Building 254

Miscellaneous Topics 256

Final Thoughts on Quantitative Models and Better Decisions 258

Appendix Calibration Tests and Answers 261

Index 273