Where are we at?

At this point we have developed an understanding on how to monetise impacts in efficient markets. However, we have yet to discuss and implement valuations for “non-market” goods and services and market distortions. Specifically, we want to be able to evaluate market failures and market distortions and capture them in the analysis. This section will look at the various ways we can identify shadow prices and implement those shadow prices in a cost-benefit analysis.

Circumstances that Require Shadow Prices

Shadow prices in cost-benefit analysis are required to ensure we capture the true social value in a cost-benefit analysis. Shadow prices are required in three situations:

• Absence of a working market – for example, there is no market for noise pollution and therefore if we are accounting for noise pollution in a CBA, we would need to estimate a price for noise.
• Market failures – these create inefficiencies in markets which results in the market price not reflecting the true value of the good or service. There are four key market failures:
  • o Public Goods
  • o Externalities
  • o Information Asymmetry
  • o Uncompetitive Market Structures
• Distortions – market distortions are due to government interventions. When the intervention is not corrective and is instead distortive, we need to adjust the prices used to value the good or service in a CBA. For example, taxes, subsidies, quotas, tariffs, price ceilings, price floors, and regulatory other interventions are all examples of distortions (covered in Chapter 8).

If we face any of these situations in our CBA, it complicates the estimation of the social prices required. Therefore, we need to adjust the willingness-to-pay or the opportunity costs to account for these price failures in the social cost-benefit analysis. The goal of this chapter it to discuss how to start the process of predicting and monetising impacts at a preliminary level, before deep diving into shadow pricing. Specifically, this chapter focuses on benefit transfers which involves drawing on the existing literature or research.

Predicting and Monetising

One of the key things we need to start with is predicting and monetising impacts in a policy, program, or project. So far, this process has been implicit. This chapter covers prediction and monetisation in a more explicit framework.

Prediction relates to the process of identifying the impacts of the project, now and into the future. This includes identifying whether impacts are considered benefits or costs, and the duration the impacts are expected to occur. Monetising is the process of quantifying the impacts in monetary terms, specifically in a common currency. Consequently, there is a difference between predicting and monetising that is essential in completing a cost-benefit analysis.

If predicting and monetising impacts is done inappropriately, the result can significantly impact the net present value (and subsequently the estimated net social benefit). Hence, we need to be aware of the three major sources of errors in a CBA that create bias:

(1) Omission Error

Omission error occurs when an impact is excluded from the CBA. If the cost or benefit excluded is substantial, it prevents the CBA from comprehensively evaluating the policy or project. When considering potential omission errors, an analyst should reflect on the size of the impact and whether exclusion of the impact would significantly change the sign of the net social benefit. This is an issue if the decision-maker is seeking to maximise the use of available resources, as omitting an important impact will introduce bias into the estimates.

Omissions errors are one of the most common errors when evaluating a policy or project – whether it occurs accidentally, or on purpose (intentional exclusions). In reality, most CBAs are incomplete because significant costs and benefits are not included in the process of predicting and monetising, leading to inaccurate and/or misleading results.

(2) Forecasting Error

Forecasting error occurs because we cannot predict the future with certainty. Each policy or project that is evaluated in a CBA has a time horizon into the future. It is difficult to know exactly what events will happen in the future. We can only truly know what has happened in the past and in the present. This shows how the use of ex-post CBA can be a useful tool in evaluating whether a past project met the goals and provides a learning opportunity for future ex-ante CBAs.

Forecasting errors also include psychological biases. Often CBA analysts can be overly optimistic in predicting and monetising impacts inflating the net social benefit.  A series of empirical studies have found there is a tendency for CBA to overestimate the social gains from large projects. It has been found that overestimation of future use benefits and underestimation of costs are common when conducting CBA (e.g. Flyvbjerg 2007). Hence, evaluation of social policies using CBA tend to be pessimistic, excluding offsetting behaviours or unintended consequences. Whereas infrastructure projects tend to be optimistic which results in the costs being underestimated.

(3) Valuation Errors

Valuation errors are caused by incorrect monetary valuation of the costs or benefits of a project. If the market price does not provide a true reflection of the value of a good or service – it is not valued correctly. Additionally, we may lack confidence in our estimates for the shadow price when we are unable to rely on a market to provide a value for the good or service. In this case to reduce valuation error, we need to ensure the shadow price is estimated appropriately. For example, if we need to value the loss of wildlife habitat, national pride, or public safety for the NPV to be correct the shadow prices must reflect the true value of these aspects in the CBA.

When attempting to provide a shadow price for a novel and relatively new good or service, there is no basis for making a confident comparison on the valuation of the shadow price. This increases the likelihood of a valuation error occurring.

Why Are Analysts Concerned with Bias?

The idea is simple. If an analyst knows where the sources of bias are coming from it is possible to anticipate the errors. This provides the opportunity to consciously “de-bias” the estimates in the project and ensure the calculated result reflects the true impact of the policy or project.

Benefit Transfers – Predicting and Monetising

Now we turn to using benefit value transfers as a method of predicting and monetising impacts. Benefit transfer is the process of using information estimated in existing, already completed studies and transferring the information into the current policy or project under evaluation.^[1] The idea of using benefit transfers is quite simple – there is no need to reinvent the wheel!

CBA analysts can take advantage of the past research efforts of others to predict impacts and utilise monetary prices for goods and services not traded on markets. Benefit value transfers involves using previously estimated shadow prices in a CBA project under consideration. These can often be called “plug-in” values (Boardman et. al., 1997). Many important shadow prices have been estimated, some multiple times – hence it may not be the best use of time and resources for an analyst to estimate the same shadow prices. This is especially useful for impacts like safety, the value of a life, and carbon emissions- shadow prices exist for these impacts due to a significant body of academic literature and research.

There are 4 essential steps to benefit transfers:

Step 1 – identify whether existing studies or values can be used as part of the transfer method.

Step 2 – decide whether the existing values from the study are appropriate for use in the CBA under consideration.

Step 3 – evaluate the quality of the research being used as part of the transfer.

Step 4 – make any appropriate adjustments to reflect the true value of the impact under consideration in the CBA.

Benefit transfers are a great starting point for identifying and valuing impacts.

Methods for Predicting

(1)   Incremental Analysis – Relative to the Status Quo

This method does not use benefit transfers, instead it builds on what was discussed in Chapter 5.

As we have discussed in the past, it is possible to conduct an incremental analysis relative to the status quo. This is one method of identifying and predicting the impacts of a policy. If impacts are the same under the status quo and the alternative policy – it is not necessary to predict and monetise these impacts as they are not expected to change. For example, suppose a retail store is considering extending its closing hours from 5pm to 6pm. There is no change in the resources used, however the incremental costs (additional wages paid) and additional benefit (additional sales made) can be accounted for in this method to simplify the prediction process.

(2)   Using Current Ongoing Policies or Projects

Similar to an in-medias res CBA, we can predict impacts of a policy or project based on trials or policies implemented by others. This is particularly useful when a policy under consideration is being trialled elsewhere. By evaluating the benefits and costs based on an ongoing policy, it provides the opportunity to determine if the policy in place should be continued, terminated, or replicated. It also serves as a basis for providing inferences about the impacts of ongoing policy. For example, “lockout laws” were trialled in Newcastle NSW. When looking at the predicted costs and benefits based on the ongoing policy in Newcastle, cities such as Brisbane and Sydney implemented and replicated the lockout laws as a method of reducing alcohol related violence. The idea of using this method allows a trial phase for the project and once data on the policy is available, we “look back (make inferences) to look forward (make predictions).”

It is important to note that when predicting impacts based on current policies, the best inference comes from experimental data with the assignment of participants into treated and control groups. This is often used when predicting the impacts of training programs (e.g., work for the dole) and welfare programs (e.g., universal basic income).

(3)   Predict On the Basis of a Similar Policy or a Series of Similar Policies

An analyst can also predict impacts based on a similar policy or a series of similar policies that are comparable to the intervention being analysed. The value of the predicted impacts of previous policies to an analyst will depend on how close the previous policy matches to the project under consideration and how well the original evaluation was conducted. When predicting impacts on a single project or a few similar projects we should ask the following questions:

1. Does the policy have the same underlying model?
2. How closely do the details of the policies conform?
3. What is the quality of the evaluation or method used in the similar policy?

It is important to note that there are sources of bias when using a single policy or a few similar policies as predictions of the effects: (1) if the research is academic in nature, we can be subject to the positive publication bias of academic journals. Often academic journals only publish positive and/or statistically significant results, (2) non-published or private sector projects are often not reported to the general public and therefore we may not have all available information on predicted impacts which may introduce bias into our assessment of the project under consideration, and (3) there may be optimism bias in the similar published policy that predicted over-optimistic results, or may have an element of bias in the impacts included in the evaluation (this comes back to the errors mentioned in Predicting and Monetising).

(4) Utilise Available Meta Analysis

Where possible, it is useful to use meta-analyses of similar policies to predict impacts of a policy under consideration. A meta-analysis is a study that uses a series of previous studies – highlighting the differences and similarities. This approach allows us to draw information from multiple resources or research to reduce the overall reported bias in predicting the impacts of the project under consideration.

Meta-analyses are useful because of the they:

• Provide a deep and comprehensive identification of relevant past research.
• Often standardise the effect size to a common metric such as dollars, percentages, or standard deviations to enable comparisons and applications across different disciplines.

Meta-analysis and Elasticities

Economics often relies on elasticities. Consequently, elasticities are often a point of evaluation for meta-analysis.  Elasticities measure of the responsiveness of a variable to changes in another variable. Having information on generic elasticities or specific elasticities will allow for prediction of impacts for a policy under consideration. In the absence of relevant policy evaluations, it is possible to predict impacts using elasticities. For example, meta-analyses are available for the price elasticity of fuel, residential water use, electricity, and cigarettes. These elasticities can be used for predicting impacts for policies related to pollution, health, etc. Elasticities have been used a lot in environmental and social interventions such as air pollution, noise pollution, environmental conservation, and drug interventions. Elasticities effectively allow an analyst to estimate the willingness-to-pay for a policy or project without starting from scratch.

Limitations of Meta-analysis

It is important to note that with meta-analysis, the result is often looking at the average of results. This may not be the best fit for the project under consideration. Often the research that matches the project under consideration best may be overlooked. Therefore, when utilizing meta-analysis as a foundation for prediction or for shadow pricing, ensure you delve deep into the cited papers.

(5)  Consult Experts

When the cost-benefit analysis under consideration is novel, it is not often possible to find existing research to support the process of predicting and monetising impacts. In these cases, it is often a good idea to consult experts in the field. These experts can make predictions based on a plausible range of outcomes based on tacit knowledge developed from their experience with the subject field.

With this method, there is likely to be bias – either overestimating or underestimating impacts as there are too many unknowns. Consulting multiple experts may be preferential, followed by deriving a mean or median prediction for inclusion in the project under consideration.

Monetising

After we have predicted the costs and benefits associated with a policy, program, or project, we need to monetise the predicted impacts. Monetising involves quantifying the predicted costs and benefits in a common currency or metric. This allows for direct comparison of the costs and benefits associated with a project. When it comes to monetisation, we deal with market and non-market costs or benefits. Market based monetisation is simple. As we have seen so far, we can use the market price as an input value in a CBA following the fundamental principles of willingness-to-pay and opportunity cost we have learned so far. When markets are distorted or missing, an analysts must find appropriate shadow prices, the problem of quantification is much more challenging.

Monetising Impacts in Missing Markets

We finish this section by looking at the use of shadow prices. It is possible to monetise impacts over the life of a project using a shadow price that was estimated by someone else. Consequently, benefit value transfers can also be used to monetise impacts. For common non-market goods and services, research exists on the appropriate shadow prices. Benefit value transfers can also be used for distorted markets, providing a useful alternative approach for estimating the value of a good or service.

Benefit value transfers can be used in the following circumstances:

(1) The absence of markets for a cost or benefit.

When a market does not exist for a good or service, we might consider the use of prior research to find appropriate shadow prices. For example, consider a road improvement project which is estimated to result in a reduction in fatalities on a section of road which averages 10 deaths per year. The problem occurs when we want to value the lives saved from the road improvement. What value would you apply to a life? There is a significant amount of research on this area that the analyst can draw upon.

(2) No appropriate market value exists due to market failure or distortions.

When the market value does not reflect the true value of the good or service, we may consider using a shadow price. For example, if we wish to estimate the value of some national parkland. The current market value of land is not reflective of the value of national park land because of the wildlife habitat, resources, or geographical location is not accounted for. Consequently, the market value therefore represents an inappropriate valuation of the parkland. However, parkland and similar recreational facilities have been widely researched and shadow prices could be borrowed as “plug-in” values for the CBA.

On a Final Note.

Take advantage of the research that is available to predict impacts and estimate shadow prices. Many important shadow prices have been estimated in the literature and often have meta-analyses available. Consider the quality of the research, the goals of the evaluation, the methodology, and whether the research is sufficiently similar to the project under consideration. It is important, when using benefit transfers for impacts or values, we should consider how closely the impact or shadow price fits the project being evaluated.

It is important to emphasize that benefit transfers can perform no better than the quality of original studies. If benefit value transfers cannot be used for shadow pricing, we can use alternative market pricing, or non-market valuation method using stated and revealed preferences. These will be covered in Chapter 12.

References

Boardman, A. E., Greenberg, D. H., Vining, A. R., & Weimer, D. L. (1997). “Plug-in” shadow price estimates for policy analysis. The Annals of Regional Science, 31(3), 299-324.

Flyvbjerg, B. (2007). Policy and planning for large-infrastructure projects: problems, causes, cures. Environment and Planning B: planning and design, 34(4), 578-597.