We investigate the effects of an audit on the choice of procurement modality by audited officials in Chile. We find clear evidence that the audit triggers a shift from auctions toward direct contracting, which goes against the goals of the national public procurement regulation that the audits are intended to enforce. We also find that there is a reduction in competition and that new and large firms are less likely to win the contract. In addition, within a subset of products for which we can compare prices and for which there is a sizeable shift toward direct contracts, audits lead to higher prices.
The shift away from auctions and toward more direct contracting surprised representatives of the Chilean Comptroller Agency. In order to look into the black box of the audit process, the Comptroller Agency agreed to conduct an additional set of similar audits in which they would collect more data on audit checks and findings at the individual contract level. Analysis of these additional audits shows that, relative to comparable direct contracts, auctions undergo more than twice as many checks, exhibit twice the number of infractions, and are twice as likely to lead to follow-up investigations. Overall, these results are consistent with the distortionary effect of an audit design that puts disproportionate scrutiny on auctions rather than direct contracts.
Public procurement is a key area for government auditing. It includes practically all public spending other than salaries and transfers and represents a large share of the economy (about 13 percent of GDP and 29 percent of total general government expenditures in the Organisation for Economic Co-operation and Development). The government is the largest buyer in many countries, and public procurement contracts can have significant impacts on supplier firms. Free and fair competition for government contracts is therefore of great importance for both the quality and cost of government purchases and to create a level playing field for all firms. For this reason, many governments and international organizations promote the use of competitive and transparent public auctions over direct contracting.
In order to analyze the impacts of government audits on procurement practices, we combine detailed administrative data on public purchases from the Chilean public procurement agency with both administrative and newly collected auditing data from the Comptroller Agency. The procurement agency provided information on all procurement processes conducted on its platform. The Comptroller Agency provided data on audits as well as on a scoring rule and scores used in the selection of entities to be audited during 2011–2012, which our analysis is based on.
These data allow us to analyze not only the impact of the audits on the share of amounts purchased through different purchase modalities but also on more nuanced aspects of the purchasing process. There is a particularly strong shift away from auctions with more than three bidders and toward the type of direct contracts that only require one quote, substantially reducing the probability that the purchase will result from a competitive process. Justifications given for the increased use of direct contracting are consistent with the overuse of direct contracting. Whenever Chilean officials use direct contracting, they need to provide one of 21 possible justifications for doing so. Almost half the increase in direct contracting in audited entities is justified as an “emergency.” This is the one justification that is known to be particularly prone to abuse.
The shift from auctions to direct contracting seems to have real economic impacts. Contracts are more likely to be awarded to small firms and to incumbent firms that have sold to the same entity before. The data also allow us to analyze impacts on prices for the small subset of products with clear and comparable units of measurement and for which there is a sizeable shift from auctions to direct contracting. Among these products, we find a substantial increase in prices. However, this result does not necessarily generalize to more complex types of purchases (such as services) for which quality criteria are typically more difficult to specify in auctions.
In order to shed light on the underlying mechanism of the shift from auctions to direct contracting, we worked with the Comptroller Agency to collect data through additional audits. These audits were structured in the same way as their standard procurement audit, except that more information was collected in the process. Auditors in Chile typically only recorded detected infractions. In these additional audits, they also recorded information on which purchase contracts were audited and which checks were conducted, independently of whether any infractions were found. Auditors in the field were not aware of the reason for this additional data collection except that the Comptroller Agency was piloting a new approach. The additional information allows us to analyze the audit process at the level of individual checks and corresponding detected infractions for each audited contract.
The finding that the additional audits show a much higher number of checks and detected infractions for auctions than for direct contracts seems to stem mainly from the fact that the audit protocol includes many more potential checks for auctions than for direct contracts. It is of course possible that the types of contracts for which auctions and direct contracting are chosen differ along observable or unobservable dimensions. We deal with this issue in two ways. First, we show that the differences in the number of checks and audit findings remain essentially unchanged when we control for contract characteristics such as the amount of the purchase and the type of product. Second, we find that most of the difference between the number of checks and detected infractions stems from auditing of the awarding stage (where the procurement process differs most between purchase modalities) rather than from the execution stage, which is very similar irrespective of modality.
When goods and services are acquired through auctions, they are thus subject to more scrutiny than when they are bought through direct contracts. More generally, this common and mechanical “auditing by checklist” approach may therefore inadvertently discourage the use of more regulated processes, which tend to involve more steps and leave a longer paper trail. If agents run the risk of making a mistake in any given step of the process, procedures involving more steps will mechanically lead to a higher probability of being found to be noncompliant. Unless penalties for infractions are lower for processes involving a longer paper trail, audits will lead to higher expected costs as a result of using more complex processes. This distortionary incentive can subvert underlying policy goals unless the regulator actually intends to discourage the more transparent process. But in many cases, the reverse is true.
We also investigate an alternative mechanism; namely, that entities might increase the use of direct contracts in year t because the likelihood of an immediate reaudit in year t + 1 might be low, and contracts awarded in year t might therefore be less subject to scrutiny. However, qualitative interviews with procurement officers and auditors revealed that they did not believe there was a reduction in the audit probability in the year following an audit, and that they felt that even if the reaudit probability in year t + 1 were zero, this would not imply that contracts from year t had a low level of scrutiny, because audits typically reach back several years to the date of the last audit. Analyzing this issue quantitatively, we find that indeed the reaudit probability is not lower after an audit. If anything, there is a slight increase in the audit probability in year t + 1.
NOTE:
This research brief is based on Maria Paula Gerardino, Stephan Litschig, and Dina Pomeranz, “Can Audits Backfire? Evidence from Public Procurement in Chile,” March 2019, https://www. econ.uzh.ch/dam/jcr:11279ded-d05c-426b-8a75-604fe07a4e42/Can-Audits-Backfire.pdf.
