Predicting from Chaos

[ghschirtz]

Niall Ferguson:
Predicting random chaos from hindsight
Why do we insist on drawing causal chains to exceedingly rare calamities after the fact?
April 23, 2007


IT WAS PREDICTABLE. Seung-hui Cho was a taciturn, moody loner. Several professors expressed concerns about the content of his work or conduct. There were complaints by two female students, a tip that he might be suicidal and a 24-hour detention in a mental health facility, but authorities said they had insufficient grounds to hold him. And guns are easy to buy in the United States. As a result, 33 people are dead.

Efforts to explain the Virginia Tech massacre perfectly illustrate one of the central points of an idiosyncratically brilliant new book by Nassim Nicholas Taleb, "The Black Swan: The Impact of the Highly Improbable." When completely caught out by some random event, we humans are wonderfully good at retrospectively predicting it. In reality, however, Cho was what Taleb calls a "black swan."

ADVERTISEMENTWhy a black swan? Taleb's starting point is what philosophers call the problem of induction. Suppose you have spent all your life in the Northern Hemisphere and have only seen white swans. You might well conclude (inductively) that all swans are white. But take a trip to Australia, where swans are black, and your theory will collapse. A "black swan" is therefore anything that seems to us, on the basis of our limited experience, to be impossible.

Over 20 years of university teaching, I have seen my share of taciturn, moody young men. Many have had difficulties with girls. Some have needed counseling. A few have required psychiatric treatment. I have often contemplated the risk that one of my depressive students might commit suicide. But the risk that one might kill 32 people? Never.

Why, Taleb asks, do we tend to confuse improbability with impossibility? Partly, he suggests, it's because evolution did not favor complex probabilistic thinking. But our flawed way of thinking also reflects the development of Western philosophy, social science and history. The Platonic school of philosophy encouraged us to prefer simple theory to messy reality; it also inclines us to select only the data that fit our theories. Taleb especially abhors the tendency of economists and others to assume that everything conforms to what is called the normal distribution, or bell curve.

Sure, says Taleb, a chart of the heights of all college students would look like a bell, with most clustered around the average height and only a negligible minority taller than 7 feet or shorter than 4 feet. But it's a mistake to look for bell curves everywhere. The statistical distributions of earthquakes, financial crises, wars and book sales, for example, obey a different set of rules.

In those cases, when you plot a chart, there is much less clustering around the average, and there are many more data points at the extremes. There are many more really big quakes, crashes, wars and bestsellers than the normal distribution would lead you to expect. I suspect the same pattern would be observed if it were possible to plot all the violent incidents that have taken place at U.S. universities in the last half-century. A few incidents would account for a huge proportion of all violent deaths.



YET IT IS Taleb's assault on traditional historiography that is most relevant here. Since Thucydides, it is true, historians have encouraged us to explain low-probability calamities (like wars) after the fact. Such storytelling helps us to make sense of a random disaster. It also enables us to apportion blame. Generations of historians have toiled in this way to explain the origins of such great calamities as, say, World War I, constructing elegant narrative chains of causes and effects, heaping opprobrium on this or that statesman.

There is something deeply suspect about this procedure, however. It results in what Taleb calls the "retrospective distortion." These causal chains were quite invisible to contemporaries, to whom the outbreak of war came as a bolt from the blue. The point is that there were umpteen Balkan crises before 1914 that didn't lead to Armageddon. Like Cho, the Sarajevo assassin Gavrilo Princip was a black swan — only vastly bigger.

The same flaw is obvious in the stories being told about the Virginia Tech massacre. If we can see the causes of Cho's rampage now, why was it not anticipated at the time? Negligence is not the only possibility. The reality is that for every one Cho who runs amok, there are thousands of depressive, misanthropic students who don't. Forcibly committing them all might avert another massacre, but the mental hospitals would be overflowing.

Perhaps the most provocative of Taleb's many provocations is his hypothesis that, as a result of globalization and the speed of electronic communication, the size and incidence of black swans may be changing. Yes, the integration of international markets seems to reduce economic volatility. But by magnifying the effects of herd-like behavior (another of our evolved traits), it also increases the tendency for winners to take all — the "Harry Potter" phenomenon — and for disasters, when they strike, to be comparably huge.

Just as there will be fewer but bigger bestsellers, Taleb argues, so there may also be "fewer but bigger crises" in the realms of finance and geopolitics. I have not quite made up my mind if the Virginia Tech massacre supports his hypothesis. But it is certainly suggestive that Cho was mimicking the behavior of the Columbine killers while at the same time exceeding their toll of victims. Now, that suggests a really chilling possibility: more and bigger black swans.


--------------------------------------------------------------------------------
nferguson@latimescolumnists.com

[swdr]

Excellent article, George. Thanks.

[DVSmith]

I forget her name, but one of the scientists from CalTech was interviewed after some notable earthquake. The interviewer asked about things like "earthquake weather". Her response was something along the lines of: Humans have a pathological need to see order in completely random events. Or something like that. I always thought that was a great line.

[Justaguy]

Originally Posted by DVSmith

I forget her name,

Dr. Lucy Jones PhD.
Geotimes - September 2004 - Profiles

I love to hear her field idiotic questions from reporters after an earthquake. They don't even realize she just called them stupid...