Fat-Tailed Distributions: Data, Diagnostics, and Dependence

This monograph is written for the numerate nonspecialist, and hopes to serve three purposes. First it gathers mathematical material from diverse but related fields of order statistics, records, extreme value theory, majorization, regular variation and subexponentiality. All of these are relevant for understanding fat tails, but they are not, to our knowledge, brought together in a single source for the target readership. Proofs that give insight are included, but for most fussy calculations the reader is referred to the excellent sources referenced in the text. Multivariate extremes are not treated. This allows us to present material spread over hundreds of pages in specialist texts in twenty pages. Chapter 5 develops new material on heavy tail diagnostics and gives more mathematical detail. Since variances and covariances may not exist for heavy tailed joint distributions, Chapter 6 reviews dependence concepts for certain classes of heavy tailed joint distributions, with a view to regressing heavy tailed variables. Second, it presents a new measure of obesity. The most popular definitions in terms of regular variation and subexponentiality invoke putative properties that hold at infinity, and this complicates any empirical estimate. Each definition captures some but not all of the intuitions associated with tail heaviness. Chapter 5 studies two candidate indices of tail heaviness basedon the tendency of the mean excess plot to collapse as data are aggregated. The probability that the largest value is more than twice the second largest has intuitive appeal but its estimator hasvery poor accuracy. The Obesity index is defined for a positive random variable X as:Ob(X) = P (X1 + X4 > X2 + X3|X1 = X2 = X3 = X4) , Xi independent copies of X. For empirical distributions, obesity is defined by bootstrapping. This index reasonably captures intuitions of tail heaviness. Among its properties, if a > 1 then Ob(X) < Ob(Xa). However, it does not completely mimic the tail index of regularly varying distributions, or the extreme value index. A Weibull distribution with shape 1/4 is more obese than a Pareto distribution with tail index 1, even though this Pareto has infinite mean and the Weibull’s moments are all finite. Chapter 5 explores properties of the Obesity index. Third and most important, we hope to convince the reader that fat tail phenomena pose real problems; they are really out there and they seriously challenge our usual ways of thinking about historical averages, outliers, trends, regression coefficients and confidence bounds among many other things. Data on flood insurance claims, crop loss claims, hospital discharge bills, precipitation and damages and fatalities from natural catastrophes drive this point home. Whilemost fat tailed distributions are ”bad”, research in fat tails is one distribution whose tail will hopefully get fatter.

---

Fat tails have entered popular discourse largely thanks to Nassim Taleb’s book The Black Swan: The Impact of the Highly Improbable (Random House, 2007). The “black swan” is the paradigm-shattering, game-changing event. But are black swans really unpredictable bolts from the blue or rather evidence of the shortcoming of traditional statistical tools?

In a new RFF discussion paper, RFF Senior Fellow Roger Cooke and Daan Nieboer highlight the unique features of “fat-tailed” phenomena. Examples of distributions exhibiting fat tails include flood damages, crop losses, natural disasters, and hospital discharge bills.

For example, prior to 2005, the most costly hurricane in the United States was Hurricane Andrew (1992) at $41.5 billion USD (2011). Hurricane Katrina set the next record—over twice as costly, at $91 billion USD (2011). This is in contrast to a “thin-tailed” distribution, such as human height (wherein the tallest human is not going to be twice as tall as the second tallest).

Prognosticators who base their predictions on historical averages will be blindsided when dealing with fat-tailed risks. As Cooke and Nieboer write, “heavy tail phenomena are not incomprehensible bolts from ‘extremistan’, but arise in many commonplace data sets. They are not incomprehensible, but they cannot be understood with traditional statistical tools. Using the wrong tools is incomprehensible.”