Imagine you have recently been named a project manager for a World Bank good governance program in Nicaragua. Your task: build a more transparent and efficient judiciary. You’ve been granted a great deal of latitude in how you allocate your resources. Recently, the Independent Evaluation Group (IEG) released a splash-making report on Cost-Benefit Analyses (CBAs) in World Bank projects. You read it and understand its straightforward message: conduct CBAs whenever possible. Do you go out of your way to please IEG, or is there a better use of your time and money?
On Tuesday May 31st CGP attended an event hosted by the Center for Global Development and SAIS that discussed precisely this question. Sadly, Andrew Warner, the author of the aforementioned IEG paper (“Cost-Benefit Analysis in World Bank Projects”), was a no-show. In his absence, Alan Gelb, a senior fellow at CGD, led the audience through a review of the paper and its implications. The paper and discussion emphasized CBAs as a tool of accountability. It seemed to us, however, that training managers and staff to be better at accountability analysis is a simpler way to improve performance in WB projects than making CBAs compulsory.
Cost-Benefit Analysis. For those of you who don’t crunch spreadsheets for fun, a CBA evaluates the net impact of a project over a specified period of time in some common denomination (usually US dollars): expected benefits minus expected costs. Sounds simple, right?
Think again. Conducting a rigorous CBA requires time, resources, and plenty of assumptions. Some of the typical questions asked are:
- Who stands to gain from the projects? Who stands to lose?
- How do you quantify (monetize in terms of dollars) gains or loses?
- What is the appropriate timeframe over which to evaluate costs and benefits?
Good CBAs are often peer-reviewed, or conduct sensitivity analysis to adjust for different valid assumptions. Both of these options consume time and resources.
In the public sector, CBAs are most useful in project appraisal. A project is justified only when a CBA finds that benefits to society will exceed the costs. In terms of World Bank policy, “whenever countries fail to pursue the alternative with the highest net present value, global resources are wasted”*.
Report Highlights. IEG’s findings reveal that the World Bank has lost its penchant for accountability tools. How, then, do we ensure we receive a bigger bang for the buck? IEG reports that:
- CBAs have fallen into disuse. According to IEG, the “percentage of projects with [CBAs] dropped from 80 percent to 25 percent between 1970 and 2008”.
- CBAs are of lower quality today than compared to previous decades. IEG reveals that only 54 percent of CBAs are currently rated as good or acceptable. In the 1990s, 70 percent of CBAs were considered of a similar standard.
What is responsible for these trends?
The report mentions several reasons. We highlight two.
Bank policies do not incentivize CBAs. Teams are preoccupied with issues that receive immediate attention from upper management. IEG evaluations—which review things like CBAs—are performed seven years after a project’s close. Why would you, sitting in Managua, spend resources on conducting CBAs in order to please an IEG evaluator seven years into the future?
Bank activities have changed dramatically. Today, it is most engaged with projects that look to improve health, governance, and education. Measuring the necessary outputs to conduct a rigorous CBA, when many of the outputs are intangible is difficult, at least relatively speaking. Measuring transparency and effectiveness in Nicaragua’s courts is, after all, much more complicated than evaluating the effect of building a pump, a road, or a port.
Team managers agree CBAs are important, but they often lack the resources to conduct good quality ones. IEG’s solution, compulsory CBAs, however, are unlikely to improve the quality of reports so long as managers are pressed for time and resources.
Conclusions. Making CBAs mandatory is no panacea; there is no evidence to suggest that it will improve project management or the quality of CBAs conducted. IEG’s proposal to make it difficult for managers to dismiss cost-benefit analysis on the basis of convenience and efficiency is good, but doesn’t go far enough. Rather than incentivizing a stream of poorly conducted CBAs, IEG should advocate for better management. WB staff should be retrained in cost-benefit analysis (and its alternatives), at the same time project managers receive resources to conduct whichever accountability analysis they wish. These two solutions, rather than mandatory CBAs alone, are much more likely to provide a lasting impact.
For the full report click here.
For the management response click here.
*(For a more thorough but brief review of cost-benefit analysis read pages 4 through 6 of “Pricing the Priceless” by Professors Heinzerling and Ackerman.)
All quotations from the original report.
Blogal Prosperity by the Center for Global Prosperity 