Such hyperbole is part of Jerry Muller’s The Tyranny of Metrics. Speaking of finance, the Catholic University of America historian criticizes the idea that “numerical acumen (premised upon probability formulas rather than empirical research) can substitute for practical knowledge about the underlying assets.” But how can “probability formulas” be excluded from empirical research? How can portfolios of complex, diversified, and abstract assets be evaluated without numbers and statistical analysis?
Muller’s case is not boosted by Oxford historian Niall Ferguson’s line that “those whom the gods want to destroy they first teach math.” Mathematics and probability theory are certainly among the tastier fruits of the Tree of Knowledge.
Yet The Tyranny of Metrics is not an attack on quantitative methods. Altogether, it is a moderate book. It only criticizes inappropriate use of metrics, metrics being defined as “numerical indicators of comparative performance based upon standardized data.” They become problematic only when “the marginal costs of assembling and analyzing the metrics exceed the marginal benefits.” Muller reminds us of the continuous importance of local knowledge à la Hayek and individual judgment.
Muller’s main argument is that inappropriate metrics corrupt the goals of public policy and incite individuals to game the system. He documents many examples of “metric fixation.” For instance, police departments classify serious crimes as minor ones in order to show better numbers on the reports they must submit to federal officials. Schools “teach to the test” in order to increase the student scores on which government largesse depends. Under then–secretary of defense Robert McNamara during the Vietnam War, body count proved largely useless as a metric of performance. Another perverse incentive is “creaming,” an example of which is surgeons declining to operate on difficult cases for fear of reducing their performance scores. And so forth.
An important aspect of Muller’s criticism relates to pay-for-performance systems in schools, hospitals, and even private companies. These systems often do not succeed in improving performance, he argues. Another aspect of metric fixation lies in the publication of such metrics in the name of transparency. The measurement of performance fails when its costs—including the opportunity cost of collecting and tabulating data—are greater than its benefits. Individuals work to increase their scores, not to do the jobs they have been hired to do.
The market and government / Muller uses theoretical insights and evidence from several fields, including economics. His command of economics is often surprising for a non-economist.
For example, he shows how the fixation on metrics can be seen as an instance of the principal–agent problem. In case of a listed corporation, the principal—the shareholders—need to make sure that the agents— the executives—maximize profits. One way to align incentives is to tie the executives’ remuneration to the metric of stock prices. The danger is that the executives will take maximizing short-run stock prices as their goal instead of the firm’s long-run discounted profits. But note how the stock market still tends to reflect the long-term value of the firm, because it is in each investor’s interest to buy a stock only if its price is lower than its expected discounted return. Market prices are not arbitrary metrics.
The problem is very different in the public sector, a difference that Muller tends to overlook. Consider a simple market—say, the market for haircuts. The barber wants to earn as much as he can in order to buy the consumption goods and services he likes. He does this by satisfying his customers. The happier they are, the more he can charge them if he offers a differentiated service, or the more customers he will get. He may—especially if he employs many people or owns a haircut chain—use metrics to measure his performance, but the real and ultimate measure lies in his profit. Moreover, he may be more entrepreneurial and rely on his intuition. At any rate, his metrics are likely to be of the sort that Muller would find reasonable.
Now consider government. If it supplies only what the market cannot efficiently supply—what economists call “public goods”—then the link between profits and consumer satisfaction is broken. People may be very happy with the national defense they get, but a large number of consumers will not voluntarily pay for it, by the very nature of a public good. Once national defense is provided, everybody can consume it equally. In this case, some metrics are required to (imperfectly) measure if taxpayers get more value than what they are forced to pay in taxes. Moreover, the process through which defense expenditures are determined and allocated must be transparent for the very reason that taxpayers are forced to finance them. Muller’s arguments against metrics and transparency become moot. As far as public goods are concerned, government inputs and outputs must be measured and transparent.
Perhaps one underlying problem in Muller’s economics is that he does not seem to believe in, or understand, consumer sovereignty. He blames “the ideology of consumer choice,” stating that “in some domains choice is particularly fraught.” But how can we assume that politicians and bureaucrats choose better? Will they have to use imperfect metrics to do this?
Government metrics / The problem is that government supplies or subsidizes a lot of services that are not public goods or, at least, not pure public goods. Instead, governments spend on education and health care, not to mention electricity, public transportation, and garbage collection. In fact, the largest part of government expenditures goes to redistribution. But the taxpayers are still forced to pay for all of this.
It is easy to understand that those taxpayers want at least to know what is spent and what the spending achieves. In these conditions, the proliferation of metrics is not surprising because it is a direct function of the extent of government intervention. This, and not metrics per se, is the problem. Shouldn’t these considerations influence Muller’s conclusions?
A related problem involves government agents, who—as James Madison noted in Federalist 51—are not angels. They will be tempted to loot the public treasury, legally or not. Even virtuous motivations are dangerous in the case of government agents because they may impose on people their own conception of the good. Government bureaucrats and politicians are paid with, and redistribute, taxpayers’ money, so they should indeed be submitted to performance metrics. Their activities should be as transparent as feasible, and they should be held accountable for what they do.
In brief, it can be argued that governments should be subject to metrics, transparency constraints, and accountability standards—and the more of them, the better—while people should be free to run their private activities as they want. It is true, as Muller conclusively demonstrates, that government-devised metrics tend to be especially inefficient, but this is in direct proportion to what the government should not be doing. He provides us with some keys to these conclusions.
Mounting regulation and metrics / There is yet another problem of government activities outside the field of public goods: mounting regulations carry benefits for some individuals and impose costs on others. One can argue that an attempt to measure these costs and benefits must be made, however difficult it is both in theory and practice. (See “The War on Consumer Surplus,” Spring 2017.) Voters must have at least the possibility of evaluating what their agents are doing. To the extent that a tyranny of metrics does exist, it is mainly caused by government interventionism, which brings us back to the real meaning of “tyranny.”
Last decade’s recession provides a good example of regulation and metrics gone wild, but Muller does not see this clearly. He blames the financial crisis on the quantification and abstraction of finance, strangely ignoring the role of government. Mortgage-based securities were pioneered in 1970 by Ginnie Mae, a federal government agency, in order to encourage the sale of residential mortgages. The Community Reinvestment Act of 1977, reinforced in the 1990s, established ratings to force banks into offering more loans and mortgages to the poor. Before the recession, financial institutions were probably the most metric-regulated businesses in America. The recession was in large part, if not ultimately, a consequence of these government-imposed requirements and metrics. (See my book Somebody in Charge: A Solution to Recessions? Palgrave Macmillan, 2011.)
Muller criticizes “short-termism” in business management, suggesting that it is partly the result of the use of performance measures in quarterly reports. But the Securities Exchange Act of 1934 forces listed companies to produce those reports. The criminalization of insider trading (using one’s private information to trade on exchanges) further encourages the use of public and transparent metrics. Government promotes short-termism in business decisions.
Nothing’s perfect, but… / Nothing is perfect of course, but government dirigisme is certainly not the least imperfect phenomenon under the sun. Muller’s case studies include the production of culture and the transmission of knowledge, from K–12 schools to colleges and universities. He rightly notes that “it is an impoverished conception of college education that regards it purely in terms of its ability to enhance earnings.”
He observes how government-imposed performance metrics damage education: “Among the stronghold of metrics in the United States has been the Department of Education, under a succession of presidents, Republican and Democratic.” Such observations should raise alerts about government’s subsidization and regulation of education.
The Tyranny of Metrics could have better analyzed the role of unbridled government in what the author calls “metrics fixation.” The problem is not metrics per se, but the fact that they are imposed by governments that should not be doing what they are doing. Government fixation is the problem. Yet Muller’s book remains an interesting one: short, unpretentious, scholarly, and full of insights. And it provokes the reader into asking further questions.
