Posts Tagged ‘statistics’


Everyone’s seen the screaming headlines: the middle-class is back!

The statistic backing up those headlines is median household income (as reported by the Census Bureau), which in 2016 was $59,039.


After years of decline, following the crash of 2000-01 and then again in 2007-08, real median household income (in 2016 dollars) has finally surpassed its previous high—of $58,665 in 1999.

But that’s not the whole story.

First, consider the fact that it took real incomes more than a decade and a half to recover from the collapse. The “good news” is not much consolation for people who endured almost two decades of zero growth in what they took home: their incomes, pensions, and wealth are permanently damaged and likely won’t be repaired within their lifetimes.

Second, the Census Bureau data show that the bulk of the gains in real income in 2016 was explained by one factor: higher employment. In other words, hours worked rose but wages did not. The members of American median households are working harder at more jobs to finally get an increase in incomes.


Finally, consider what is measured in those headline numbers. The median, as folks might remember from a statistics course (or just a teacher’s explanation of how they grade), is the “middle” value: half above, and half below. But we can also calculate the mean (or “average” value) and compare the two. As is clear from the chart, while both the median and mean values (the green and red lines in the chart, measured on the left, respectively) have reached all-time highs, the gap between them—the “skew” in the distribution—has also grown over time. In fact, the ratio of the mean and median incomes (the blue line, measured on the right) has increased—from 1.23 in 1980 to 1.41 in 2016.

This is a clear indication that, while median household incomes in the United States have finally recovered from the crises of recent years, the middle-class itself is falling further and further behind those at the top.

Wouldn’t it be useful if those income statistics were reported in the headlines!


Regular readers know I take statistics quite seriously. So, as it turns out, did Stephen Jay Gould who, in the most poignant story about statistics of which I am aware, explained how important it is to go beyond the abstractions of central tendencies and understand the distribution of variation within the numbers.

And right now, when the numbers are under attack—when, for example, the new Trump administration is threatening to purge the inconvenient numbers about climate change—it is even more important to understand the role statistics play in economic and social life.*

William Davies [ht: ja] offers one story about statistics, starting with the recent populist attacks on public statistics and the questioning of the experts that produce and interpret them. His view is that, for all their faults, the numbers collected and disseminated by technical experts within national statistical offices need to be defended—as the representation of “common ideas of society and collective progress”—against the rise of private “data.”

A post-statistical society is a potentially frightening proposition, not because it would lack any forms of truth or expertise altogether, but because it would drastically privatise them. Statistics are one of many pillars of liberalism, indeed of Enlightenment. The experts who produce and use them have become painted as arrogant and oblivious to the emotional and local dimensions of politics. No doubt there are ways in which data collection could be adapted to reflect lived experiences better. But the battle that will need to be waged in the long term is not between an elite-led politics of facts versus a populist politics of feeling. It is between those still committed to public knowledge and public argument and those who profit from the ongoing disintegration of those things.

I understand the threat posed by big, private data—all those numbers that are collected “behind our backs and beyond our knowledge” when we travel, make purchases, and participate in social media, and in turn are utilized to sell us even more commodities (including, of course, political candidates).

But I also think Davies, in his rush to condemn private control over big data, presents too uncritical of a defense of “the kinds of unambiguous, objective, potentially consensus-forming claims about society that statisticians and economists are paid for.”

Consider, for example, one of the “unambiguous, objective, potentially consensus-forming claims about society” Davies himself cites: GDP. Just last Friday, the headlines reported that the U.S. economy grew “only” 1.6 percent during the last quarter of 2016, “the lowest level in five years.”

The presumption was that the decline in the number (with respect to both previous quarters and economists’ forecasts) represented a fundamental problem. But why should it—why should a decline in the growth rate of GDP be taken as a sign of something that needs to be fixed?

Davies does mention that GDP “only captures the value of paid work, thereby excluding the work traditionally done by women in the domestic sphere, has made it a target of feminist critique since the 1960s.” But the controversies surrounding that particular statistic are much more widespread than Davies would have us believe. As a number of recent books (including Ehsan Masood’s The Great Invention: The Story of GDP and the Making and Unmaking of the Modern World) have clearly explained, the initial formulation of that particular measure of national income as well as subsequent revisions have involved theoretical and political choices about what should and should not be included—government expenditures but not labor within households, the production of fossil fuels but not the destruction of the natural environment, sales of private security but not the growing inequality it is designed to protect against.**

Even more fundamentally, GDP is a measure of market transactions, of goods and services produced—and thus the contemporary counting of the elements celebrated by Adam Smith’s notion of the “wealth of nations.” But what it doesn’t measure are the conditions under which those commodities are produced.

Me, I’d be much more willing to join forces with Davies and defend the claims about society that statisticians and economists are paid for if they were also paid to calculate and publicly report one other number, S/V, the rate of exploitation.


**We should remember that perhaps the real hero of volume 1 of Capital was Leonard Horner, who as a factory inspector “carried on a life-long contest, not only with the embittered manufacturers, but also with the Cabinet, to whom the number of votes given by the masters in the Lower House, was a matter of far greater importance than the number of hours worked by the ‘hands’ in the mills.”

**Other useful books on GDP include the following: Philipp Lepenies’s The Power of a Single Number: A Political History of GDP (Columbia University Press, 2016), Lorenzo Fioramonti’s Gross Domestic Problem: The Politics Behind the World’s Most Powerful Number (Zed Books, 2013), and Thomas A. Stapleford’s The Cost of Living in America: A Political History of Economic Statistics, 1880-2000 (Cambridge University Press, 2009).


Mark Tansey, “Coastline Measure” (1987)

The pollsters got it wrong again, just as they did with the Brexit vote and the Colombia peace vote. In each case, they incorrectly predicted one side would win—Hillary Clinton, Remain, and yes—and many of us were taken in by the apparent certainty of the results.

I certainly was. In each case, I told family members, friends, and acquaintances it was quite possible the polls were wrong. But still, as the day approached, I found myself believing the “experts.”

It still seems, when it comes to polling, we have a great deal of difficult with uncertainty:

Berwood Yost of Franklin & Marshall College said he wants to see polling get more comfortable with uncertainty. “The incentives now favor offering a single number that looks similar to other polls instead of really trying to report on the many possible campaign elements that could affect the outcome,” Yost said. “Certainty is rewarded, it seems.”

But election results are not the only area where uncertainty remains a problematic issue. Dani Rodrik thinks mainstream economists would do a better job defending the status quo if they acknowledged their uncertainty about the effects of globalization.

This reluctance to be honest about trade has cost economists their credibility with the public. Worse still, it has fed their opponents’ narrative. Economists’ failure to provide the full picture on trade, with all of the necessary distinctions and caveats, has made it easier to tar trade, often wrongly, with all sorts of ill effects. . .

In short, had economists gone public with the caveats, uncertainties, and skepticism of the seminar room, they might have become better defenders of the world economy.

To be fair, both groups—pollsters and mainstream economists—acknowledge the existence of uncertainty. Pollsters (and especially poll-based modelers, like one of the best, Nate Silver, as I’ve discussed here and here) always say they’re recognizing and capturing uncertainty, for example, in the “error term.”


Even Silver, whose model included a much higher probability of a Donald Trump victory than most others, expressed both defensiveness about and confidence in his forecast:

Despite what you might think, we haven’t been trying to scare anyone with these updates. The goal of a probabilistic model is not to provide deterministic predictions (“Clinton will win Wisconsin”) but instead to provide an assessment of probabilities and risks. In 2012, the risks to to Obama were lower than was commonly acknowledged, because of the low number of undecided voters and his unusually robust polling in swing states. In 2016, just the opposite is true: There are lots of undecideds, and Clinton’s polling leads are somewhat thin in swing states. Nonetheless, Clinton is probably going to win, and she could win by a big margin.


As for the mainstream economists, while they may acknowledge exceptions to the rule that “everyone benefits” from free markets and international trade in some of their models and seminar discussions, they acknowledge no uncertainty whatsoever when it comes to celebrating the current economic system in their textbooks and public pronouncements.

So, what’s the alternative? They (and we) need to find better ways of discussing and possibly “modeling” uncertainty. Since the margins of error, different probabilities, and exceptions to the rule are ways of hedging their bets anyway, why not just discuss the range of possible outcomes and all of what is included and excluded, said and unsaid, measurable and unmeasurable, and so forth?

The election pollsters and statisticians may claim the public demands a single projection, prediction, or forecast. By the same token, the mainstream economists are no doubt afraid of letting the barbarian critics through the gates. In both cases, the effect is to narrow the range of relevant factors and the likelihood of outcomes.

One alternative is to open up the models and develop a more robust language to talk about fundamental uncertainty. “We simply don’t know what’s going to happen.” In both cases, that would mean presenting the full range of possible outcomes (including the possibility that there can be still other possibilities, which haven’t been considered) and discussing the biases built into the models themselves (based on the assumptions that have been used to construct them). Instead of the pseudo-rigor associated with deterministic predictions, we’d have a real rigor predicated on uncertainty, including the uncertainty of the modelers themselves.

Admitting that they (and therefore we) simply don’t know would be a start.


Special mention

download 181704_600


The Social Security system, now in its 81st year, is (according to the Congressional Budget Office [pdf]) solvent until 2029. In that year, the trust fund will be exhausted—but, still, even without any changes, the program will be able to pay out at least 71 percent of mandated benefits. And all it would take is eliminating the earnings cap (currently $118,500) to make the program fully financed for the foreseeable future.

But you wouldn’t know that from Wharton economist Olivia S. Mitchell, who like many others who have attempted to propose “reforms” to the system attempts to gin up the numbers. Her particular version is to get people to delay taking their monthly payments by promising them a lump-sum payout at the later date.

The problem, of course, is we should be doing exactly the opposite—lowering the retirement age and expanding benefits (which we could do by eliminating the earnings cap).

Even more, as Michael Hiltzik explains, Mitchell cites a particular statistic in order to create the specter of a system facing imminent crisis, to which she can then offer a “painless solution.”

The questionable part of her article appears near the bottom, where she writes:

“The Social Security shortfall is enormous. Actuaries have estimated that it’s on the order of $28 trillion in present value. That’s twice the size of the gross domestic product of the U.S. So a small delay in claiming won’t solve the problem. We’re also going to have to change the benefit formula. We’re going to have to make changes in the retirement age.” (Emphasis added.)

Most Social Security experts view that $28-trillion figure as a red flag. That’s because many people who cite it are ideologues aiming to scare the public into thinking the program’s finances are far worse than they really are. Let’s see what makes the statistic, and Mitchell’s use of it, so misleading.

The figure is an estimate of the present value of Social Security’s unfunded obligation not as it exists today, but as if it were calculated out to infinity. Economists find the so-called infinite horizon model useful in some contexts. But as it’s typically applied to Social Security it’s beloved by ideologues because it produces a really big, and really scary, estimate of the accumulated deficit.

The infinite projection appears in the annual Social Security Trustees Report, but its placement there is controversial. The Social Security Advisory Board’s 2015 technical panel of economists, actuaries and demographers recommended dropping the infinite projection from the trustees reports altogether, for two reasons. One is that it incorporates enormous uncertainties. Estimating costs, revenues and policy changes for Social Security’s conventional 75-year forecasts is hard enough; the influences playing on the program hundreds or thousands of years into the future are literally unimaginable. That makes the infinite projection “unhelpful as a guide to policy-making,” the panel reported.

The second reason is that it’s so vulnerable to misinterpretation. As an earlier technical panel observed, the projection is sometimes “quoted in policy discussions without including its relation to corresponding GDP, which is both misleading and shifts the focus from more useful metrics.”

Interestingly, that’s exactly what Mitchell does. (We should mention in passing that Mitchell actually gets her numbers wrong. The infinite projection deficit, as published by the trustees in their most recent report, was $25.8 trillion as of Jan. 1, not $28 trillion; U.S. GDP, according to the Bureau of Economic Analysis, was $18.2 trillion as of the end of 2015, not $14 trillion as Mitchell implies.)

In her MarketWatch article, Mitchell doesn’t disclose that the figure she’s citing is the infinite projection, which could lead some readers to think she’s talking about Social Security’s current deficit. (In current terms, Social Security actually runs an annual surplus and is expected to do so until 2020.) Even worse, she juxtaposes it with current gross domestic product by stating that it’s “twice the size” of GDP today. The unwary reader might be led to think that a Social Security “crisis” is on the verge of bankrupting the U.S. in the here and now. . .

Mitchell’s lump-sum plan might be a useful element in a Social Security fix if it were entirely clear that a fix was necessary. But the fact that she relied on an exaggerated statistic to make her case suggests that there may not be such a strong case, after all.

The CBO’s projection of the 75-year actuarial deficit of the Social Security program as a share of GDP is only 1.45 percent—not nearly as dramatic as Mitchell’s statistic. It’s a number that doesn’t conjure up crisis or induce panic. All it suggests is that “tweaking” the system (by, as I suggested, raising the earnings cap) will make the program solvent and create the space for what we should really be doing, lowering the retirement age and expanding benefits for American workers.

Lies, damned lies, and Mitchell’s statistic serve a very different purpose.


As everyone knows (or should know), the United States is an international outlier when it comes to incarceration rates.

According to the Hamilton Project,

The U.S. incarceration rate—defined as the number of inmates in local jails, state prisons, federal prisons, and privately operated facilities per every 100,000 U.S. residents—is more than six times that of the typical Organisation for Economic Co-Operation and Development (OECD) country. A variety of factors can explain the discrepancy in incarceration rates. One important factor is higher crime rates, especially rates of violent crimes: the homicide rate in the United States is approximately four times the typical rate among the nations shown in this chart. Additionally, drug control policies in the United States—which have largely not been replicated in other Western countries—have prominently contributed to the rising incarcerated population over the past several decades. Another important factor is sentencing policy; in particular, the United States imposes much longer prison sentences for drug-related offenses than do many economically similar nations.

What that means if there are more than 2 million Americans in prison or jail.

Here’s how the incarceration rate has changed over the past 100-plus years of U.S. history:


A baseball player made entirely out of their statistics

Special mention