Two recent economic studies purporting to estimate the impact of the Trans-Pacific Partnership (TPP) agreement on the U.S. economy have sparked a kerfuffle between the deal’s advocates and detractors. One study, published by the Peterson Institute for International Economics, estimates increases to U.S. income of 0.5 percent by 2030 with gains to labor accruing slightly more than gains to capital. The other, published by Tufts University’s Global Development and Environment Institute, estimates that the TPP would reduce U.S. income by 0.5 percent, reduce employment by almost half a million jobs, and increase income inequality. The findings of each study are being trumpeted as dispositive by their respective constituencies. Who’s right?

In a recent blog post, PIIE-affiliated economist Robert Lawrence wrote that to judge the credibility of these models, three questions should be asked: Is the model used appropriate for exploring trade policy? Does the model depict TPP sensibly? Are the results credible? Lawrence then goes on to explain why he answers “yes” to each question regarding the PIIE study and “no” to each regarding the Tufts study. Well sure, Bob, at a minimum, those criteria are important. And they help distinguish the PIIE model as relatively credible – that is, relative to the Tufts model. But what about relative to reality?

A model might depict TPP sensibly, but incompletely and imprecisely. How can we be sure those imperfections don’t have a large impact on the results? And even if the results are credible, in that they don’t deviate dramatically from expectations, their purpose – or, at least, the weight assigned to these studies in the public’s mind – is to produce reasonable estimates, not to corroborate the model’s capacity to process reasonable expectations.

With apologies to my trade economist friends, anyone who treats the estimates produced by economic models as mathematical truths is, well, part of the problem. Lawrence doesn’t do that, but too many trade policy combatants do. Certainly, some models are more rigorous than others, but all rely on assumptions. The greater the number and complexity of exogenous policy changes being modeled, the greater the number of estimates and assumptions to incorporate, and the further removed from reality the results will be. Sometimes the estimates are merely best guesses and sometimes the assumptions have no better than a 50 percent probability of occurrence. For example, many of the economic benefits of TPP will derive from reductions in non-tariff barriers to trade, such as regulatory opaqueness. How does one model the increase in regulatory transparency? How does one account for stricter environmental or labor or intellectual property regulations? How does one assign numeric values to rules limiting restrictions on cross-border data flows?

The PIIE study assumes that 75 percent of non-tariff barriers constitute trade restrictions (with 25 percent being regulations intended to promote or protect social goods, such as food quality and safety, for example) and that about 75 percent of those NTBs affecting goods trade and 50 percent affecting services trade are actionable. So, the PIIE assumptions are that 56.25 percent and 37.50 percent of NTBs affecting good and services, respectively, will be reformed under the TPP, and the effect of each reform is modeled as a trade cost reduction of 0 to 100 percent for each combination of industry and country. One other assumption is that 20 percent of these reforms also will benefit non-TPP members, which – through a feedback loop – will magnify the benefits to TPP-country economies. So, modeling the TPP requires numerous assumptions and estimates from the outset.

With respect to the PIIE study, perhaps, we can accept that the sign of the coefficient on income is positive (“the TPP will produce benefits for the U.S. economy”), but the magnitude of 0.5 isn’t much better than a guess because it is the product of so many assumptions and estimates. And despite the model’s relative rigor in using microeconomic behavioral equations to transmit the effects of TPP, predicting how economic agents will react to a confluence of policy changes is a dicey proposition – a fatal conceit. Economics is not science. The preferences, information, and knowledge that drive economic decisions are all personal, nuanced, and dispersed. They cannot be represented mathematically with any precision. Assumptions about how economic agents should react that are, at best, based on probabilities and expectations of utility- or profit-maximization don’t always hold. To their credit, the Tufts economists, responding to Lawrence’s legitimate indictment of their approach, concede that economic models involve a lot of guesswork and that economic modelers have a long way to go.

What the public and policymakers should be considering – what should be under the spotlight – are the rules of the TPP, not the projected outcomes. The outcomes cannot be known with certainty. The rules are objective and concrete. We should be able to draw conclusions about the desirability of the TPP from its language – from the rules it articulates – without guarantees of particular outcomes. The TPP should be judged by the degree of economic freedom it restores, not by a shouting match over highly contestable estimates. Indeed, some chapters of the TPP are expressly about reducing trade barriers, including tariffs and other obstacles to competition. Those provisions should be universally embraced, as they will help restore our economic freedoms. (For those whose companies or industries are subsequently exposed to greater competition when barriers are removed, please realize that you have been benefitting at everyone else’s expense and that liberalization rights that wrong.)

Other chapters of the TPP are less about liberalization and more about crafting common rules about how governments treat foreign enterprises and how they enforce labor rights, environmental regulations, intellectual property provisions, and so on. It is less clear whether and how these “governance” chapters enhance or impair or economic freedom. But each chapter can be assessed exhaustively on a qualitative basis, without need of highly malleable estimates of economic outcomes.

In the weeks and month ahead, look for a comprehensive, chapter-by-chapter assessment of the TPP from my Cato Institute trade policy colleagues and me.