Filtered by author: On Balance Blog Editor Clear Filter

On Balance: Using Retrospective Analysis to Increase Policy Learning in Europe

A retrospective exercise is an opportunity to learn and improve. With this in mind, an international consortium led by CSIL (Centre for Industrial Studies) recently developed an evaluation framework to carry out a retrospective assessment of infrastructure projects in several environmental sectors. The framework was developed as part of a study being carried out on behalf of the European Commission (Directorate-General for Regional and Urban Policy) to look retrospectively at 10 of the major infrastructural projects co-financed by the Commission over the period 2000 to 2013. While it will come as no surprise that, in retrospect, forecasts are imperfect, learning from mistakes or unexpected outcomes may improve the quality of forecasts. In turn, better forecasts may lead to better policy-making. This post reports some preliminary results from the study; more complete results will be presented at the Society for Benefit-Cost Analysis Conference in March 2019.

 

Just to give you a flavor of the study, the 10 projects are briefly presented in the table below. All the projects have been in operation for at least 5 years, placing—de facto–the assessment in an intermediate viewpoint when compared to the projected lifetime of the infrastructure (typically 20 to 30 years).

Read More

On Balance: Can Cost Benefit Analysis Tell Us If Our City Should Host the Olympics? It Does. But Could Do It Better.

There is growing skepticism among both academics and government officials about the benefits of large-scale sport and cultural events. Although Input-Output (IO) has long been the dominant approach to estimating the impacts of these events, the method faces criticism for both its lack of realism and the incompleteness of its results. Consequently, economists have begun to turn to two alternative approaches: computable general equilibrium (CGE) and cost-benefit analysis. These approaches can take into account effects not captured within an IO framework. They also often produce strikingly different results than those obtained using an IO model. This post reports some preliminary results from ongoing research evaluating nearly 60 studies that use cost-benefit methods to evaluate events. More complete results will be presented at the Society for Benefit-Cost Analysis Conference in March 2019.


An IO approach is fundamentally a multiplier approach, converting expenditures (on infrastructure, for example) into gross production in the host city. One limitation is that IO treats expenditures, such as those for infrastructure or by the local population, as fully additional; since this approach does not take account of substitution (i.e., that these resources have been diverted from other uses), it may overstate the favorable economic impact of an event. Another concern is IO’s focus only on gross domestic product and other related impacts, rather than on welfare, which also recognizes externalities and opportunity costs (Massiani 2018).

The earliest examples of the use of cost-benefit analysis and CGE to evaluate mega-events appeared in the 1980s and 1990s, respectively. Since then, almost 60 cost-benefit analyses of mega events of different sizes have been conducted. (For an examination of the use of CGE in evaluating mega-events, see Massiani (2018a).)

Read More

On Balance: Review of The Cost-Benefit Revolution by Cass R. Sunstein

Fomenting revolution brings to mind crowds storming the barricades, not analysts struggling to debug a spreadsheet or craft a clear sentence. Yet in his book, The Cost-Benefit Revolution, Cass Sunstein (Robert Walmsley University Professor, Harvard Law School) argues that benefit-cost analysts are doing exactly that. The barricades we surmount are decision-making errors resulting from overreliance on intuition and emotion, our weapons are science and economics, and our achievements are better policies.

As a previous Administrator of the Office of Information and Regulatory Affairs in the U.S. Office of Management and Budget, which oversees the U.S. regulatory program, it is not surprising that Sunstein focuses on the use of benefit-cost analysis in the regulatory realm. However, his insights and conclusions are broadly applicable to wherever benefit-cost analysis is practiced.

Those of you who are familiar with Sunstein’s scholarship will recognize familiar themes. He integrates and builds on previous work, including articles initially published in the Journal of Benefit Cost Analysis: “The Value of a Statistical Life: Some Clarifications and Puzzles” and “Cost-Benefit Analysis, Who’s Your Daddy?.” Among these themes, the most central are the findings from behavioral science which indicate that our intuition often leads us astray. Sunstein argues that benefit-cost analysis is an important corrective, promoting more rational decisions. He reminds us that we can agree or disagree on the underlying theory while still agreeing that the results provide useful and important information about the consequences of alternative policy choices.

Read More

On Balance: Two quick fix solutions to baseline estimation challenges: good enough for practical analysis?

Since 2010, I have been a practitioner of practical benefit-cost analysis at Environment and Climate Change Canada (ECCC), working with colleagues to inform policymakers and stakeholders about the likely impacts of proposed federal environmental regulations. I would like to examine a key issue in developing estimates of the benefits and costs of these regulations: the development of the baseline in a benefit-cost analysis.

In recent years, a number of articles in the Journal of Benefit Cost Analysis have addressed the question of how best to conduct—and evaluate—a regulatory benefit-cost analysis. Two of these subsequently resulted in posts to On BalanceConsumers Guide to Regulatory Impact Analysis, by Susan Dudley, and Two Decades of Benefits and Costs: Promise and Pitfalls, by Clark Nardinelli. Another article by Richard Morganstern, Retrospective Analysis of U.S. Federal Environmental Regulation, examines various estimates for environmental rules and analyzes issues relevant to developing credible baselines.

These articles remind us that baselines in a regulatory impact analysis can be slippery: in estimating benefits and costs the comparison is not between before-and-after regulation but between a counterfactual with-and-without regulation. In turn, the “counterfactual” should take into account the effect of other regulations, the evolution of the market in the absence of regulation, and other external factors. In short: developing this counterfactual requires understanding how the baseline has been changing and how it is likely to change.

Read More

On Balance: What’s the Score? The Congressional Budget Office and its Role in the Policy Process

The Congressional Budget Office (CBO) plays an important role in the federal legislative process. CBO’s score on a given bill—that is, its estimate of how it would affect the federal budget deficit—can determine whether Congress decides to go forward with the bill, modify it to get a more favorable estimate, or simply drop it. Given their importance, debates over CBO’s scores and the methods they use to produce them can be as controversial as the bills that are being considered. While this controversy can be politically motivated (with advocates on either side of an issue arguing for a score that is more favorable to their position), it also stems from limited understanding of CBO’s intended role in the process—and reflects the difficulty of conducting analyses of benefits and costs in the context of policy decisions.

 

Because Congress wanted an objective scorekeeper to develop accurate budget estimates, it created CBO in the Congressional Budget and Impoundment Control Act of 1974. Among other functions, CBO is required by law to produce a formal cost estimate for nearly every bill that is approved by a full committee of either the House or the Senate. It also produces informal cost estimates for a much larger number of legislative proposals. This scorekeeping function has become more important over time. CBO’s Update to the Budget and Economic Outlook estimates the current federal budget deficit at almost $700 billion and projects it will more than double over the next decade, and Congress is acutely conscious of the budgetary implications of potential legislation.

Read More

On Balance: Using Cost-Benefit Analysis to Make the World a Better Place

If you had billions of dollars to make the world a better place, and could spend it however you wanted, how would you maximize your impact? This is not just a thought experiment, but a daily concern for the world’s governments, philanthropists, and multilateral institutions. Since 2004, Copenhagen Consensus has been using cost-benefit analysis to help decision makers identify highly effective interventions. While most cost-benefit analysis seeks the best solution for a single problem, Copenhagen Consensus takes a wider view, looking across all major domains for policies that improve social welfare.

 

Our approach is, first, to identify smart ideas across 15 to 20 relevant policy domains by consulting widely in the region or country in which we’re working. This process typically generates more than a thousand ideas. An advisory council then whittles this list down to around 100—with preference given to ideas that are likely to be highly cost-effective or those backed with significant funding and political will. We then work with some of the world’s top economists to analyze the costs and benefits of these 100, insofar as data and other constraints permit.

Read More

On Balance: Nudging Electricity Consumption

Nudges are all the rage in behavioral economics and public policy applications around the world because of their potential for doing good at little or no cost. Economists, including Kip Viscusi (2018), have begun examining previously unexplored sides of nudges including some involving energy consumption. An often employed nudge in the area of conserving energy and reducing pollution is the home energy report, where after a home energy survey a household receives a message, usually monthly, that compares its energy use to that of neighbors and suggests ways to reduce electric or gas use and, in turn, its carbon footprint. Each month the consuming unit can also see how its energy use compares to its own past usage too. The subtleties of the benefits of home energy surveys and home energy reports is the subject of a recent article, "Differential and Distributional Effects of Energy Efficiency Surveys: Evidence from Electricity Consumption," available open access at the Journal of Benefit-Cost Analysis.

 

Nobel Laureate in Economic Sciences, Sir Angus Deaton (2018), has emphasized the importance of moving beyond only the mean differences revealed in the “gold standard” random controlled trials and recommended other statistical setups to consider effect heterogeneity including intervention distribution effects, such as how the treatment effect differs by predicted outcome quantile. The particular intervention my co-author, Dr. Galib Rustamov, and I investigated involves about 4200 customers of an Investor Owned Utility in California who voluntarily participated in a Home Energy Effectiveness Survey in January 2009 and January 2010. In addition to energy use the survey contained more than 130 questions, including household size, income, house size, and energy using/saving behavior and appliances. The comparison group contained customers who did not participate in the surveys then but would voluntarily participate in subsequent years. Our estimation procedures included quantile difference-in-differences propensity score matching regressions of two years of monthly energy consumption.

Read More

On Balance: Review of "Cost Benefit Analysis" by Per Olov Johansson and Bengt Kristrom

Cambridge Elements has created a series on Public Economics, edited by Robin Boadway, Frank Cowell and Massimo Florio. This is part of a major project by Cambridge University Press, which is intended to provide peer-reviewed analytical surveys and frontier topics in all the disciplines. We happily note that the first published “Element” in this series is Cost Benefit Analysis, by Per-Olov Johansson and Bengt Kriström (2018). Cost-Benefit Analysis is available as a free download for a limited time, and is for sale (relatively inexpensively) in print at Cambridge and at online booksellers, such as Barnes and Noble. The Element is just above eighty pages (plus a short technical annex and a long list of references).

Encompassing the whole of Cost-Benefit Analysis in such a reduced volume is a tour de force, as acknowledged by the authors in the prologue. The aim is to provide a comprehensive and non-technical state of the art overview of Cost-Benefit Analysis, including both basic classical results and some new frontiers. It has been successfully reached.

The authors’ task is made easier by a book they co-authored in 2015, Cost Benefit Analysis for Project Appraisal, which was about three times as long as the Element. While of course the authors build on this source, the Element has several differentiating features.

Read More

On Balance: Executive Order 12866 and the Durability of Core Regulatory Principles

On September 24, 2018, experts gathered at the George Washington University (GW) to commemorate the 25th anniversary of Executive Order (E.O.) 12866Regulatory Planning and Review—and discuss the implications of its provisions and future prospects. The Society for Benefit-Cost Analysis (SBCA) cosponsored the event with the GW Regulatory Studies CenterABA Section of Administrative Law and Regulatory Practice, and the Trachtenberg School of Public Policy and Public Administration. (Additional information on the event, including commentaries by the speakers, and videos when they become available, are posted on the GW website.)

 

The E.O., which President Clinton signed in 1993, expanded on previous executive orders requiring benefit-cost analysis and formalized the principles of regulation and centralized review procedures that still guide the rulemaking process (see also an earlier “On Balance” post, written by Susan Dudley and Clark Nardinelli, which preceded the event). The three panels highlighted insights from government experts, scholars, and past and present administrators of the Office of Information and Regulatory Affairs (OIRA) responsible for applying the order’s principles under presidents Clinton, Bush, Obama, and Trump. The most significant theme that emerged from each panel was the durability of E.O. 12866’s principles and process—both across administrations and over time.

Read More

On Balance: Two Decades of Benefits and Costs: Promise and Pitfalls

A new article in the Fall Issue of the Journal of Benefit-Cost Analysis (JBCA),  “Some Pitfalls of Practical Benefit-Cost Analysis,”  describes common pitfalls that well-meaning analysts fall into. Over the past 23 years I have worked on and supervised hundreds of benefit-cost analyses. Most of these analyses have dealt with public health regulations proposed by the Food and Drug Administration, although on occasion I have reviewed analyses from other government agencies and academia. I’ve seen these pitfalls occur many times and at one time or another I’ve been guilty of most of them.

 

Benefit-cost analysis can have great value in creating efficient regulations and in making the effects of regulations and policies accessible to policy makers and the public. From public investments to traffic control to educational choice, benefit-cost analysis has shown its value. Several articles in the JBCA over the years have highlighted “good practices” in specific areas where benefit-cost analysis is done, notably papers by Susan Dudley (with others), and by Scott Farrow and Kip Viscusi.

Read More

On Balance: Reflecting on the Longevity of Executive Order 12866 and Looking to the Future

September 2018 marks the 25th anniversary of Executive Order (EO) 12866, which requires U.S. federal agencies, “in deciding whether and how to regulate, [to] assess all costs and benefits of available regulatory alternatives, including the alternative of not regulating.” It further states that, “in choosing among alternative regulatory approaches, agencies should select those approaches that maximize net benefits (including potential economic, environmental, public health and safety, and other advantages; distributive impacts; and equity), unless a statute requires another regulatory approach.”

Since it was signed on September 30, 1993, four different presidents with markedly diverse regulatory philosophies have relied on EO12866 to guide both the procedures and analytical practices for developing new regulations.

To commemorate the 25th anniversary of EO 12866, the Society for Benefit-Cost Analysis  is co-sponsoring a forum with the George Washington University (GW) Regulatory Studies Center , the Section of Administrative Law and Regulatory Practice  of the American Bar Association, and the Trachtenberg School of Public Policy & Public Administration at GW. The forum, to be held on the GW campus on the afternoon of September 24, will feature panels addressing the past, present, and future of EO 12866.

Read More

On Balance: Reporting back: the World Congress for Environmental and Resource Economists in Gothenburg, Sweden

Gothenburg, Sweden hosted the 2018 World Congress for Environmental and Resource Economists, which met from June 25-29, 2018. The five-day conference was held at the School of Business, Economics, and Law, which is part of the University of Gothenburg. The conference was made possible by the joint effort of three major economics associations: the Association of Environmental and Resource Economists (AERE), the East Asian AERE (EAAERE), and the European AERE (EAERE).

 

To put into perspective the scale (and importance) of this conference, consider that this conference, which occurs only once every four years, attracted over 1,500 participants, with a program that included more than 1,000 papers, 30 policy sessions, 70 thematic sessions, a large poster luncheon, and six plenary sessions. During the parallel sessions, I and many others found ourselves struggling with the happy conundrum of having to choose among several high-quality concurrent sessions.

Read More

On Balance: Review of “Teaching Benefit-Cost Analysis” by Scott Farrow

As the feasibility of using benefit-cost analysis (BCA) as a practical tool of policy analysis has increased, so too has the need for materials to aid those of us who are called upon to teach BCA. Teaching Benefit-Cost Analysis: Tools of the Trade, edited by Scott Farrow (Professor of Economics, University of Maryland, Baltimore County), is a distinctive and welcome addition to the collection of such materials.

 

The volume is part of the Elgar Guides to Teaching series from Edward Elgar Publishing. It differs from traditional textbooks on BCA in that teachers of benefit-cost analysis, rather than students, are the intended audience. The book comprises nineteen chapters, which are organized into two thematic sections; one section focuses on broad conceptual issues in undertaking benefit-cost analysis, and the other contains entries on specific issues that arise in implementing different types of BCA.

Read More

On Balance: Retrospective Analyses Are Hard: A Cautionary Tale

Under the 1990 Clean Air Act Amendments, the U.S. Environmental Protection Agency (EPA) was required to establish standards limiting air toxics emissions from industrial plants. An article in the latest issue of the Journal of Benefit Cost Analysis  (JBCA)“Retrospective Analyses Are Hard: A Cautionary Tale from EPA's Air Toxics Regulations,” takes a retrospective look at 5 of the largest rules issued by EPA in the initial round of air toxics rulemaking over the period 1995 to 2000. For the rules examined, our estimates suggest mixed results, in terms of the reductions in emissions that were achieved. However, our efforts during the project also reinforce the difficulty of obtaining adequate plant emissions data, even where there is an established database--in this case, the Toxic Release Inventory (TRI).

 

The project reported in this paper is part of the broader Regulatory Performance Project (RPP) launched by Resources for the Future. Since 2014, RPP has funded nine retrospective analyses covering a total of 34 cost-benefit or cost-effectiveness comparisons from a highly diverse set of environmentally oriented rules, in order to compare observed outcomes for these rules with a range of credible baselines and ex ante estimates.  A recent article by Richard Morgenstern in the JBCA looks more broadly at these nine studies.

Read More

On Balance: Taking Benefit-Cost Analysis on the Road

One of the reasons I look forward each year to the annual conference of the Society for Benefit Cost Analysis is connecting with fellow economists who have chosen careers in my specialty, which has evolved to be regulatory policy and analysis. I enjoy sharing and hearing everyone’s war stories and appreciate the advice I receive from those who have “been there, done that” and hope that I add value when I offer the same to new economists working to influence the policy process.

 

In April, I had the opportunity to engage with other regulatory economists in a very different setting, when I represented the Department of Transportation (DOT) as part of a U.S. delegation to Bogota, Colombia. The purpose of the trip was to inform Colombia’s efforts to implement good regulatory practices by sharing U.S. regulatory experience. In preparing for the trip, I learned that the adoption of good regulatory practices is part of the accession requirements of the Organization of Economic Cooperation and Development (OECD), which Colombia is seeking to join. Also, Colombia recently implemented a mandatory requirement to conduct Regulatory Impact Analysis (RIA) for new regulations and its regulatory agencies are in the early phases of complying with that requirement.

Read More

On Balance: Communicating about Benefit-Cost Analysis

It is challenging to explain benefit-cost analysis (BCA) to the general public or to members of other professions lacking sufficient knowledge of—and appreciation for—applied microeconomics. This post discusses the nature of this challenge, and raises the question: should the SBCA develop a more concerted communications strategy to better explain the fundamental concepts of the discipline and their practical application?

 

Part of the communication challenge faced by BCA practitioners is the one shared by any epistemic community. A specialized terminology and shared paradigm increase communication efficiency within the group, at the cost of communicating outside of it. This communication gap is especially consequential for the BCA field, however, because the purpose of BCA is to inform public decision-making. To do that, BCA practitioners have to communicate with other professionals involved in the policymaking process who may have their own disciplinary paradigms (e.g., scientists, engineers, and lawyers), as well as journalists and opinion-leaders, and of course, the general public. The nature of the BCA communication challenge is multi-faceted. First, terminology like “benefits,” “costs,” and “efficiency” have precise definitions within the discipline but are popularly used with a range of other meanings. Conceptual differences underlying the same terminology are common—such as the public perception that the term “benefit” applies to economic development objectives like “job creation” (Courant 1994) or to the receipt of transfer payments. Increased governmental revenue, or regional income associated with infrastructure projects, are commonly seen as benefits and represented as such in practitioner-produced “benefit-cost” analyses (Boardman et al. 1993). Beyond differences in language usage are differences in the way policy is perceived. The public often sees policy through the lens of stakeholder self-interest, rather than from a larger concept of the “public interest” (Krutilla 2005). From a typical stakeholder’s perspective, the economic efficiency measure—the sum of a policy’s net effects on everyone -- is less salient than are commonly-stated political objectives, such as better air quality, small business growth, job creation, and the like. Heterogeneous stakeholders are not likely have a unanimous consensus about policy choices and this well-known reality, coupled with the fact that the benefits and costs of a public decision can fall on different groups, will create winners and losers. Stakeholder losses, unless widely diffused, are likely to be more salient than the relatively abstract notion that a portfolio of efficient policies and/or income transfers will probably leave everyone ultimately better off. Moreover, stakeholder conflict over public decision-making is likely to induce a “thinking fast” (Kahneman 2011) mindset disinclined to accept the logic and results of evidence-based analysis.

Read More

On Balance: Review of “Pricing Lives: Guideposts for a Safer Society” by W. Kip Viscusi

Pricing Lives: Guideposts for a Safer Society  (Princeton University Press, 2018) is a tour de force. It provides an entertaining, accessible account of the modern approach to valuing mortality risk. In non-technical prose, it covers the major aspects of the approach and how it should be applied to social decisions.  It shows how US regulatory agencies’ adoption of the ‘value per statistical life’ (VSL) to replace the ‘cost of death’ (i.e., human capital) approach has led to more-protective regulation and argues the VSL approach should be extended beyond regulation, to private-sector decisions about product design and to government sanctions for regulatory violations. It also finds that the values used outside the US are typically far too small; revising these upward would lead to more-protective and more-appropriate regulation in other countries.

 

Over the course of several chapters, the author, W. Kip Viscusi (Vanderbilt University), examines the rationale and controversy surrounding the variation in VSL with individuals’ characteristics (especially age and income), with the cause of death (e.g., traumatic injury, lingering disease), and the source of the hazard (e.g., occupational, consumer-product defect or lack of safety features, terrorism) and the implications of these differences for public policy. He discusses issues as diverse as the difference between ‘identified’ and ‘statistical’ lives and detecting and correcting for effects of publication bias when seeking to develop a value from the literature. For evaluating risk equity, he proposes using a VSL-like measure (equality of the marginal costs of risk reduction across people).

Read More

On Balance: Report from the SBCA President on the 2018 Annual Conference and The Plenary Presentation by Tomas Philipson

It was my honor to preside over the 2018 Society for Benefit-Cost Analysis (SBCA) Conference held March 14 – 16 at the George Washington University in Washington, DC. I am happy to report that by all indications it was another successful conference. We had 319 total registrants from 19 countries across 5 continents. U.S. participants represented 31 states and D.C. Half of all participants were affiliated with the U.S. federal government. The next largest group were academics (almost one-third); other attendees were from the private sector or from state and international government agencies. The three pre-conference professional development workshops were well attended and also drew a mix of participants from federal, state, and international agencies, as well as academics.

 

The conference evaluations we’ve received so far rank the overall quality of the 2018 SBCA conference at 4.4 on a scale of 1 to 5, with 5 being the highest. We are combing through the details and responses to the open-ended questions to see how we can do even better next year. We welcome more feedback, even when open-ended comments point out problems we can’t do much about, like the unseasonably cold weather in D.C. this past March!

Read More

On Balance: Application of BCA in Europe

Despite the obvious attraction of Benefit-Cost Analysis (BCA) for policy evaluation, its implementation rate varies across countries and sectors. Although the concept of BCA can be traced back to European thinkers, it was first applied in the United States. The evaluation by the U.S. Army Corps of Engineers’ of the U.S. Flood Control Act of 1936 is often regarded as the first use of BCA, but President Ronald Reagan’s Executive Order 12291 issued in 1981 provided considerable impetus to its use in the United States. Although perceptions of BCA as anti-regulatory caused some adversaries of BCA to argue for its elimination in policy making, BCA is now viewed also as a tool to promote regulation.

 

Recent years have witnessed growing demand for economic appraisals of policies in different sectors in Europe, but the implementation rate is still low compared to that in the United States. The United Kingdom is considered the early adopter of BCA in Europe with appraisals of transport projects in the early 1960s. At the European level, represented by the European Commission (EC) within the European Union (EU), while BCA was used for policy evaluation in the early 1990s, the rate of use was very low. More recently, however, the application of BCA has risen, accompanied by the availability of national guidelines and manuals on how to conduct BCA, as well as by the development of the EC’s Guide to Cost-Benefit Analysis of Investment Projects for Cohesion Policy 2014-2020, which evolved from a brief document to a comprehensive document (now in its 5th version) backed by EU legislation.

Read More

On Balance: Don't Undervalue Life

Proper application of the value of a statistical life (VSL) is essential to preventing the systematic undervaluation of life throughout the world. The values used by U.S. federal agencies to monetize prospective risk reductions formerly were too low but have increased over time and are now in a range consistent with the economic literature. However, the values agencies assign to fatalities in setting regulatory sanctions are extremely low—often a fraction of the current estimates for VSL estimates used in regulatory contexts. Further, other countries still monetize risk reductions using techniques that undervalue life relative to VSL estimates. 

 

The origin of monetizing mortality risks in the United States and other countries can be traced to application of the human capital approach, which some agencies termed the “cost of death.” In the view of agency officials, life was too sacred to value, so instead they assigned a monetary value to the deaths. When I introduced the VSL in 1982 to resolve the dispute between the Occupational Safety and Health Administration (OSHA) and the Office of Management and Budget over the proposed hazard communication standard, the shift to the VSL as the measure of mortality benefits boosted benefits by a factor of 10. The bolstering of benefit assessments through the use of the VSL reflects the historical role of the VSL in providing an economic efficiency rationale for benefit assessments, making regulations more stringent than if set based on cost-of-death values.

Read More