Where Are We At?

At this point we have developed an understanding of how to monetise impacts in efficient markets. However, we have yet to discuss and implement valuations for non-market goods and services, and market distortions. Specifically, we want to be able to evaluate market failures and market distortions, and capture them in our analysis. This section will look at the various ways we can identify shadow prices and implement those shadow prices in a cost-benefit analysis.

Circumstances that Require Shadow Prices

Shadow prices in cost-benefit analysis are required to ensure we capture the true social value in a cost-benefit analysis. Shadow prices are required in three situations:

• Absence of a working market – for example, there is no market for noise pollution and, therefore, if we are accounting for noise pollution in a CBA, we would need to estimate a price for noise.
• Market failures – these create inefficiencies in markets which results in the market price not reflecting the true value of the good or service. There are four key market failures:
  • Public goods
  • Externalities
  • Information asymmetry
  • Imperfectly competitive market structures
• Distortions – market distortions occur due to government interventions. When the intervention is not corrective and is instead distortive, we need to adjust the prices used to value the good or service in a CBA. For example, taxes, subsidies, quotas, tariffs, price ceilings, price floors, and other regulatory interventions are all examples of distortions (covered in Chapter 8).

If we face any of these situations in our CBA, this complicates our valuation of the good or service as we must estimate a shadow price by adjusting the willingness-to-pay or the opportunity costs to account for these price failures in a social cost-benefit analysis. The goal of this chapter it to discuss how to start the process of predicting and monetising impacts at a preliminary level, before deep diving into shadow pricing. Specifically, this chapter focuses on benefit transfers, which involves drawing on the existing literature and research.

Predicting and Monetising

One of the key things we need to start with is predicting and monetising the impacts of a policy, program, or project. So far, this process has been implicit. This chapter covers prediction and monetisation using an explicit framework.

Prediction relates to the process of identifying the impacts of the project, now and into the future. This includes identifying whether impacts are considered benefits or costs, and the duration over which those impacts are expected to occur. Monetising is the process of quantifying the impacts in monetary terms, specifically, in a common currency. Consequently, understanding the difference between predicting and monetising is essential to complete a cost-benefit analysis.

If predicting and monetising impacts is done inappropriately, the result can significantly impact the net present value (and subsequently, the estimated net social benefit). Hence, we need to be aware of the three major sources of errors that create bias in a CBA:

(1) Omission error

Omission errors occur when an impact is excluded from a CBA. If the cost or benefit excluded is substantial, it prevents the CBA from comprehensively evaluating the policy or project. When considering potential omission errors, an analyst should reflect on the size of the impact and whether exclusion of the impact would significantly change the sign of the net social benefit. This is an issue if the decision-maker is seeking to maximise the use of available resources, as omitting an important impact will introduce bias into the estimates.

Omissions errors are one of the most common errors when evaluating a policy or project – whether it occurs accidentally, or on purpose (intentional exclusions). In reality, most CBAs are incomplete because significant costs and benefits are not included in the process of predicting and monetising, leading to inaccurate and/or misleading results.

(2) Forecasting error

Forecasting errors occur because we cannot predict the future with certainty. Each policy or project that is evaluated with an ex-ante CBA has a time horizon into the future. It is impossible to know exactly what events will happen in the future. We can only truly know what has happened in the past and the situation in the present. This shows how the use of ex-post CBA can be useful for evaluating whether a past project met its goals and providing learning opportunities for future ex-ante CBAs.

Forecasting errors also include psychological biases. Often CBA analysts can be overly optimistic when predicting and monetising impacts, inflating the net social benefit.  A series of empirical studies have found there is a tendency for CBA to overestimate the social gains from large projects. It has been found that overestimation of future use benefits and underestimation of costs are common when conducting CBA (e.g. Flyvbjerg 2007). Hence, evaluation of social policies using CBA tend to be pessimistic, excluding offsetting behaviours or unintended consequences. Whereas infrastructure projects tend to be optimistic which results in the costs being underestimated.

(3) Valuation error

Valuation errors are caused by incorrect monetary valuation of the costs or benefits of a project. If the market price does not provide a true reflection of the value of a good or service – it is not valued correctly. Additionally, we may lack confidence in our estimates of the shadow price when we are unable to rely on a market to provide a value for the good or service. In this case, to reduce valuation error, we need to ensure the shadow price is estimated appropriately. For example, if we need to value the loss of wildlife habitat, national pride, or public safety for the NPV to be correct, the shadow price must reflect the true value of all these aspects in the CBA.

When attempting to provide a shadow price for a novel and relatively new good or service, valuation is more challenging as there is little existing research to identify an accurate shadow price. This increases the likelihood of a valuation error occurring.

Why are analysts concerned with bias?

The idea is simple. If an analyst knows where the sources of bias are coming from, it is possible to anticipate the errors. This provides the opportunity to consciously “de-bias” the estimates in the project and ensure the calculated result reflects the true impact of the policy or project.

Benefit Transfers – Predicting and Monetising

Now we turn to using benefit value transfers as a method of predicting and monetising impacts. Benefit transfer is the process of using existing estimates from already completed studies and transferring that information into the current policy or project under evaluation.^[1] The idea of using benefit transfers is quite simple – there is no need to reinvent the wheel!

CBA analysts can take advantage of the past research efforts of others to predict impacts and monetise the value of goods and services not traded on markets. Benefit value transfers involve using previously estimated shadow prices in a CBA of a project under consideration. These can often be called “plug-in” values (Boardman et. al., 1997). Many important shadow prices have been estimated, some multiple times – hence it may not be the best use of time and resources for an analyst to estimate the same shadow prices. This is especially useful for impacts like safety, the value of a life, and carbon emissions – shadow prices already exist for these impacts thanks to the significant body of academic literature and research on these topics.

There are four essential steps to benefit transfers:

Step 1 – Identify whether existing studies or monetary values can be used as part of the transfer method.

Step 2 – Decide whether the existing values from the study are appropriate for use in the CBA under consideration.

Step 3 – Evaluate the quality of the research being used as part of the transfer.

Step 4 – Make any appropriate adjustments to reflect the true value of the impact under consideration in the CBA.

Benefit transfers are a great starting point for identifying and valuing impacts.

Methods for Predicting

(1)   Incremental analysis relative to the status quo

This method does not use benefit transfers, instead it builds on what was discussed in Chapter 5.

As we have discussed in the past, it is possible to conduct an incremental analysis relative to the status quo. This is one method of identifying and predicting the impacts of a policy. If impacts are the same under the status quo and the alternative policy – it is not necessary to predict and monetise these impacts as they are not expected to change. For example, suppose a retail store is considering extending its closing time from 5pm to 6pm. There is no change in the resources used, however the incremental costs (additional wages paid) and additional benefit (additional sales made) can be accounted for in this method to simplify the prediction process.

(2)   Using current ongoing policies or projects

Similar to an in-medias-res CBA, we can predict impacts of a policy or project based on trials or policies implemented by others. This is particularly useful when a policy under consideration is being trialled elsewhere. By evaluating the benefits and costs based on an ongoing policy, it provides the opportunity to determine if the policy in place should be continued, terminated, or replicated. It also serves as a basis for providing inferences about the impacts of ongoing policy. For example, “lockout laws” were trialled in Newcastle, New South Wales. After looking at the predicted costs and benefits based on the ongoing policy in Newcastle, cities such as Brisbane and Sydney implemented and replicated those lockout laws with the aim of reducing alcohol related violence. This method allows for a trial phase for the project and once data on the policy is available, we “look back (make inferences) to look forward (make predictions).”

It is important to note that when predicting impacts based on current policies, the best inferences come from data gathered in experiments where participants were assigned into treatment and control groups. This is often used when predicting the impacts of training programs (e.g., Work for the Dole) and welfare programs (e.g., universal basic income).

(3)   Prediction on the basis of a similar policy or a series of similar policies

An analyst can also predict impacts based on a similar policy or a series of similar policies that are comparable to the intervention being analysed. The value of the predicted impacts of previous policies to an analyst will depend on how close the previous policy matches to the project under consideration and how well the original evaluation was conducted. When predicting impacts on a single project or a few similar projects we should ask the following questions:

1. Does the policy have the same underlying model?
2. How similar are the details of the policies?
3. What is the quality of the evaluation or method used in the similar policy?

It is important to note that there are sources of bias when using a single policy or a few similar policies to predict the effects: (1) if the research is academic in nature, we can be subject to the positive publication bias of academic journals. Often academic journals only publish positive and/or statistically significant results, (2) non-published or private sector projects are often not reported to the general public and therefore we may not have all available information on predicted impacts which may introduce bias into our assessment of the project under consideration, and (3) there may be bias such as optimism bias in existing research which may impact the quality of results (this comes back to the errors mentioned in Predicting and Monetising).

(4) Utilise available meta-analysis

Where possible, it is useful to use a meta-analysis of similar policies to predict impacts of a policy under consideration. A meta-analysis is a study which evaluates a series of previous studies – highlighting the differences and similarities. This approach allows us to draw information from multiple resources or research to reduce the overall bias when predicting the impacts of the project under consideration.

Meta-analyses are useful because of the they:

• Provide a deep and comprehensive identification of relevant past research.
• Often standardise the effect size to a common metric such as dollars, percentages, or standard deviations to enable comparisons and applications across different disciplines.

Meta-analysis and elasticities

Economic analysis often relies on elasticities. Consequently, elasticities are often a point of evaluation for meta-analysis.  Elasticities measure the responsiveness of a variable to changes in another variable. Having information on generic elasticities or specific elasticities helps us predict the impacts of a policy under consideration. In the absence of relevant policy evaluations, it is possible to predict impacts using elasticities. For example, meta-analyses are available for the price elasticity of fuel, residential water use, electricity, and cigarettes. These elasticities can be used for predicting the impacts of policies related to pollution, health, etc. Elasticities have been used a lot in environmental and social interventions related to air pollution, noise pollution, environmental conservation, and drug interventions. Elasticities allow an analyst to effectively estimate the willingness-to-pay for a policy or project without starting from the very beginning.

Limitations of meta-analysis

It is important to note that with meta-analysis, the result is often looking at the average of results. This may not be the best fit for the project under consideration. Often the research that matches the project under consideration best may be overlooked. Therefore, when utilizing meta-analysis as a foundation for prediction or for shadow pricing, ensure you delve deeply into the cited papers.

(5)  Consult experts

When the cost-benefit analysis under consideration is novel, it is often not possible to find existing research to support the process of predicting and monetising impacts. In these cases, it is often a good idea to consult experts in the field. These experts can make predictions about a plausible range of outcomes based on tacit knowledge developed from their experience with the subject field.

With this method, there is likely to be bias, either overestimation or underestimation of the impacts, as there are too many unknowns. Consulting multiple experts may be preferential, followed by deriving a mean or median prediction for inclusion in the project under consideration.

Monetising

After we have predicted the costs and benefits associated with a policy, program, or project, we need to monetise the predicted impacts. Monetising involves quantifying the predicted costs and benefits in a common currency or metric. This allows for direct comparison of the costs and benefits associated with a project. When it comes to monetisation, we deal with market and non-market costs and benefits. Market based monetisation is simple. As we have seen so far, we can use the market price as an input value in a CBA following the fundamental principles of willingness-to-pay and opportunity cost that we have learned so far. When markets are distorted or missing, an analysts must find appropriate shadow prices which makes the problem of quantification much more challenging.

Monetising impacts in missing markets

We finish this section by looking at the use of shadow prices. It is possible to monetise the impacts over the life of a project using a shadow price that was estimated by someone else. Consequently, benefit value transfers can also be used to monetise impacts. For common non-market goods and services, research exists on their appropriate shadow prices. Benefit value transfers can also be used for distorted markets, providing a useful alternative approach for estimating the value of a good or service.

Benefit value transfers can be used in the following circumstances:

(1) The absence of markets for a cost or benefit

When a market does not exist for a good or service, we might consider the use of prior research to find appropriate shadow prices. For example, consider a road improvement project which is estimated to result in a reduction in fatalities on a section of road which averages ten deaths per year. The problem occurs when we want to value the lives saved from the road improvement. What value would you apply to a life? There is a significant amount of research on this area that the analyst can draw upon.

(2) No appropriate market value exists due to market failure or distortions

When the market value does not reflect the true value of the good or service, we may consider using a shadow price. For example, if we wish to estimate the value of some national parkland, the current market value of the land would not reflect the value of its use as a national park because the wildlife habitat, resources, or geographical location are not fully accounted for. Consequently, the market value therefore represents an inappropriate valuation of the parkland. However, parkland and similar recreational facilities have been widely researched and shadow prices could be borrowed as “plug-in” values for the CBA.

On a Final Note.

Take advantage of the research that is available to predict impacts and estimate shadow prices. Many important shadow prices have been estimated in the literature and often have meta-analyses available. Consider the quality of the research, the goals of the evaluation, the methodology, and whether the research is sufficiently similar to the project under consideration. When using benefit transfers for impacts or values, it is important  to consider how closely the impact or shadow price fits the project being evaluated.

It is important to emphasize that benefit transfers can perform no better than the quality of the original studies that they come from. If benefit value transfers cannot be used for shadow pricing, we can use alternative market pricing, or non-market valuation methods using stated and revealed preferences. These will be covered in Chapter 12.

References

Boardman, A. E., Greenberg, D. H., Vining, A. R., & Weimer, D. L. (1997). “Plug-in” shadow price estimates for policy analysis. The Annals of Regional Science, 31(3), 299-324.

Flyvbjerg, B. (2007). Policy and planning for large-infrastructure projects: problems, causes, cures. Environment and Planning B: planning and design, 34(4), 578-597.