Tuesday, October 9, 2007

Black Swan

I've noticed a bit of a buzz surrounding the book "The Black Swan: The Impact of the Highly Improbable" by Nassim Nicholas Taleb. I haven't read it but I think I understand the premise -- which asserts that straight probability and statistical analyses break down outside of a closed world assumption. In other words, when one considers that something odd can happen outside of the scope of your well understood reality, the infinite possibilities out there can bring about some surprising new eventualities. The classic case that Taleb brings up concerns the case of a previously undiscovered Australian black swan, of which European naturalists predicted would never occur, as the genetic probability of only white swans occurring amounted to 100%. But of course, the new species of black swan turned the old paradigm on its ear and the conventional math wisdom proved pointless.

Due to the popularity of The Black Swan book, a certain species of cornucopian tends to think the swan allegory portends optimism for our oil future. I first saw it here on TOD.

I assert that instead of a Black Swan representing new discoveries of oil, we have here a Black Passenger Pigeon. As I understand it, because of a large population of swans, you could get mutations leading to the evolution of a black species. I would consider this rare but mathematically possible. But it would seem vanishingly small to assume the possibility that a black passenger pigeon would suddenly appear as the overall population gets decimated and eventually becomes extinct. And this has nothing to do with a renewable resource like birds, but rather that we have pretty much scoped out every hiding area on the earth. See the Dispersive Discovery model to back this up. (note that the original TOD commenter said "couldn't follow I'm afraid" to my counter-allegory)

By the same token, the idea that large oil reservoirs get discovered first presents an optical illusion of sorts, if not another inverse Black Swan. The unlikely possibility of a huge new find hasn't as much to do with intuition, as to do with the fact that we have probed much of the potential volume. And large finds occur at the peak of the dispersively swept volume.

Looking at some of the Amazon reviews for The Black Swan, I have to agree with the author on the misapplication of the Normal distribution for many situations.
The whole point is that traditional stat, econ, finance techniques are mostly around the first moment (mean) but the distributions in finance tend to be non-normal and it's the risk that we should pay more attention to. That's a point few people would disagree with. What the author may not have known is that there are stat techniques out there that handle all the issues mentioned - while it's true that there's a lot of room for improvement, it's misleading to say that this is an area ignored by the academics and practitioners.
According to other reviews, the author disses math and religion in equal amounts -- something that displeases that wide a range of people must have some rhetorical substance and probably a worthwhile read (his earlier book "Fooled by Randomness" looks interesting as well).



As I type this I have got on the radio Sam Seder subbing for Mike Malloy talking to author Michael Klare, the author of several oil depletion books, including "Blood and Oil". Sam skewered Freddie Thompson and his presidential debate assertion that we have plenty of oil left.