Is the Mother of All Black Swans Coming?

dragon kingWhether you like Nassim Taleb’s writing style or not, his books and the ideas within are worth careful study.

For example, a key theme of Nassim Taleb’s “The Black Swan” is to pay close attention to the possibility of the rare event, the outlier that brings devastating consequence. Taleb forcefully and intelligently argues that these outliers are much more common than we think (1 in 100 year floods happen every 3-5 years) especially when we base our statistical analysis on the assumptions of a normal distribution of data and independence.

However, some interesting research by Didier Sornette at the Swiss Federal Institute of Technology shows that while power laws can explain some of the extreme events that have taken place in history, there are some extreme outliers that even defy classic power laws.

A post in MIT’s Technology Review blog cites the following:

“Sornette gives as an example the distribution of city sizes in France, which follows a classic power law, meaning that there are many small cities and only a few large ones. On a log-to-log scale, this distribution gives a straight line–except for Paris, which is an outlier and many times larger than it ought to be if it were to follow the power law.”

The article also mentions that the city of London also follows this same example.

Sornette calls these extreme outliers “Dragon Kings.” A sobering commentary from the article ensues; “(The) seemingly ubiquitous presence of these dragon kings in all kinds of data sets means that extreme events are significantly more likely than power laws suggest.”

This in turn suggests that the Mother of all Black Swans might be unaccounted for in your data set. And if this is the case, does this mean that we cannot predict these extreme events,  and/or that preparation is futile?

I would love to hear your thoughts!

Advertisements

4 comments

  1. I may be naïve, but how could one sample over 36000 (the number of French cities) dismiss a distribution assessment on any statistical test?

    Thus, is it statistically an outlier?

    “many times larger than it ought to be” is not very clear. I would say twice too larger at worst. Not big deal anyway.

    But of course, your comment is still right.

  2. Sebastien, thank you for taking the time to comment. The examples of London and Paris were cited in the article, however I suspect these are just samples and that there are many others not yet known/uncovered – especially if we look outside the category of “cities”. I found the research a bit scary in that there are some outliers that are even defying power laws – thus “the mother of all Black Swans” comment.

  3. Interesting, I feel scary when an human behaviour fits too much a statistical law (I have a machine learning background).

    Nevertheless, black swans are a real mess when it comes to prediction. In such cases, domain knowledge will beat any model. Human can still beat machine learning here.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s