This blog series is a partnership of the Society for Benefit-Cost Analysis On Balance blog and Dr. Zoë Plakias’ Spring 2021 Benefit-Cost Analysis (AEDECON 5330) class at The Ohio State University. Students interviewed experts in benefit-cost analysis to learn about what they do and why they do it. All interviews have been edited for brevity and clarity with the help of Dr. Plakias and are shared with the approval of the interviewer and interviewee.

• Expert: Lisa A. Robinson
• Interviewer: Katherine Bowman

Lisa A. Robinson is Deputy Director of the Center for Health Decision Science and Senior Senior Research Scientist at the Center for Health Decision Science and Center for Risk Analysis at the Harvard T.H. Chan School of Public Health. She served as the President of the Society for Benefit-Cost Analysis in 2014.

Q: How did you get interested in benefit-cost analysis?

A: After I graduated from the Harvard Kennedy School, I went to work in Washington, D.C. Although we’d read many case studies that described how policy decisions were made, I was surprised to see how often they appeared arbitrary. Benefit-cost analysis seemed like a much-needed antidote. It promotes a systematic and objective approach to exploring impacts.

As my career progressed, my interest deepened. When we moved back to Boston, I went to work at Industrial Economics, Incorporated (IEc), a consulting firm. I led several regulatory impact analyses for the U.S. Environmental Protection Agency (EPA) and others and began getting involved in drafting guidance. One reason for my increasing interest was my experience with the practical implications of this work. Theoreticians often focus on the conceptual underpinnings of benefit-cost analysis: measuring net social welfare, applying the Kaldor-Hicks potential compensation test, and so forth. While these principles are important, they are not the primary reason why benefit-cost analysis is useful in my view. Decisions are rarely, if ever, based solely on the results of the analysis. Significant impacts often cannot be quantified and decision-makers must also consider legal, political, budgetary, and other constraints.

What is much more important is that the analytic framework promotes rigorous investigation of the positive and negative impacts of the policy options as well as clear communication of the results to policymakers and other stakeholders. We may find, for example, that a policy that initially appears desirable is infeasible. The needed technology may not be available or may be prohibitively expensive. Or we may find that the link between the policy and the hoped-for benefits is very tenuous. The scientific evidence may be weak or inconsistent; people may be unlikely to react to the policy as expected or hoped.

Another reason why I have stayed involved in this field (over what is now almost 40 years) is that the work is endlessly challenging. Policymakers typically can’t wait long for more or better information. They often need to make decisions based on what data are now available or can be collected within a few months. That means we are constantly forced to think carefully about how to appropriately use the available information and how to clearly communicate the implications of uncertainties. It also means that thinking ahead and collecting data and developing models so that they are available to address future policies is very important. EPA’s BenMAP program is an excellent example of this type of initiative.

We also frequently work under tight time and budget constraints, which requires us to think carefully about how to best prioritize our work. What impacts are most in need of further investigation, either because associated uncertainties significantly affect the estimates of net benefits or because they are of great interest to stakeholders? For example, while decreased mortality risks dominate the benefit estimates for air pollution policies, it seems like a mistake to ignore the effects on asthma. Asthma-related benefits may be small, but they are important in people’s day-to-day lives. Another challenge is that the policy options under consideration often change as the decision-making process unfolds. How can we design the analysis so that we can easily pivot to address these new options?

Becoming an expert in benefit-cost analysis also means that you have an opportunity to work across numerous policy areas, which in turn means there is always something new to learn. I’ve been involved in assessing policies ranging from environmental protection to health and human services to homeland security. More methodological development is also always needed. For example, as I moved into academia, I’ve had the opportunity to explore the implications of behavioral economics and happiness (subjective well-being) research, to investigate approaches for assessing employment impacts, and to consider the distribution of costs and benefits across advantaged and disadvantaged groups. I’ve also investigated how to encourage greater use of benefit-cost analysis in low- and middle-income as well as high-income settings. There are endless interesting and important topics to investigate.

Q: What's one piece of advice you have for students interested in benefit-cost analysis? What is a lasting impact that you'd like to leave with them?

A: For students, what seems most important is to develop a range of skills, ranging from statistical analysis to communicating effectively with the general public. Combining academic training with hands-on experience is also essential. Benefit-cost analysis in the real world is often much messier than the textbook version, due to limited data as well as time and budget constraints. Learning how to appropriately and rigorously operate within these constraints is crucial, and is best mastered by working with an experienced mentor. David Weimer’s benefit-cost analysis course at the University of Wisconsin’s LaFollette School has always intrigued me, because he requires students to work on a team project on a real issue for an actual client, providing invaluable experience.

In terms of lasting impacts, what is most gratifying is when students tell me that what they have learned in class permeates other aspects of their lives. They begin thinking more systematically about the benefits and costs of their personal decisions as well as the decisions that are the focus of their professional lives, whether in business, government, or academia.

Q: What are some key trends that you see and where do you see them going?

A: It’s been wonderful to watch benefit-cost analysis move from an approach initially used predominantly to assess major regulations in high-income countries to an approach applied in numerous policy areas globally, a trend that I hope and expect will continue. The MacArthur Foundation, who funded the founding of the Society for Benefit-Cost Analysis and its Journal of Benefit-Cost Analysis, played an essential role in expanding the use of benefit-cost analysis in social policy; more recently, they joined forces with the Pew Charitable Trusts to expand this work. The Bill and Melinda Gates Foundation is playing an important role in encouraging the use of benefit-cost analysis in low- and middle-income countries. Many international organizations, such as the World Bank and the OECD, are also promoting its use.

Another major trend is the increasing focus on addressing distribution. Although almost every major guidance document emphasizes the importance of providing information on who gains and who loses to supplement the estimates of net benefits, our work suggests this guidance is often ignored. In the U.S., the Biden Administration’s new executive orders emphasize the need to pay attention to this issue. Although the Biden executive orders (here and here) basically reiterate what's been in executive orders at least since 1981, people are treating them as something new -- not realizing that the real problem is actually doing the work. The requirements need to be enforced. It's not enough to just say distribution is important. You must assess it.

Q: Do you feel that working in industry, government, or academia suits you better?

A: I’ve been very fortunate to work in government, consulting, and academia. I’ve enjoyed it all. Each has its advantages and disadvantages, and draws on different skill sets and different personality traits.

The most exciting thing about being in government is you have a very direct impact on policy. The disadvantage is that you are working in a large bureaucracy and have to deal with the politics.

In consulting, the type of work you do and your impact on policy depends on your clients as well as the focus of the firm. You often have the opportunity to work on large, longer term research projects with significant real-world impacts, but are a step removed from the actual decision-making process. The challenge is to do good work and to find intellectually engaged clients who understand and promote that work. Consulting is often misunderstood as being paid to do whatever a client tells you to. Successful consultants are more proactive. They recognize that their success depends on the quality of their work as well as their relationship with clients. They work with the client to understand their needs and to help them be effective.

In academia, your work can be highly conceptual and theoretical, far removed from practical application. Or it can involve assessing current policy choices and substantially influencing decisions. It really depends on where you are and what you do. One advantage is that the focus on protecting academic freedom can help shield you from those who wish to influence or bias your work. A disadvantage is you will have the greatest ability to pursue your interests as a tenured faculty member, but such positions may be hard to find. They also often involve responsibilities other than research, such as teaching and committee work, that may or may not be of interest. Regardless, the choice of employment really depends on determining what suits you best.

Q: How has the pandemic changed work you're doing and how you're doing it?

A: I and others who do this type of work have been very lucky in that we could continue to work from home. The COVID pandemic also drew substantial media attention to the practice of benefit-cost analysis and the valuation of mortality risk reductions. Early in the pandemic, I read a lot of terrible analyses because people who had never done this type of work before quickly recognized that it provided important insights into the choices we faced. Over time, the quality of the analyses and the general public’s understanding of the methods improved, in part due to the efforts of many more experienced researchers. As a result, there is much better understanding of the use and usefulness of benefit-cost analysis.