Energy efficiency is frequently promoted as the lowest cost “resource” to achieve consumer cost reductions, improve the economy and reduce environmental impacts. One challenge with energy efficiency is measuring it. To quantify the impact of an energy efficiency device, program or portfolio, a “but-for calculation” is required. What would have been the energy consumption, costs and environmental implications but-for the energy efficiency initiative?
There has and is a long-standing dispute in the literature regarding the cost-effectiveness of energy efficiency. The point of this post is not to explain, let alone resolve, that disagreement. Instead, it is to propose an evaluation framework that is likely to determine the impacts of any given energy efficiency initiative.
The bottom line is that without objective, independent and timely evaluations, energy efficiency initiatives will not achieve meaningful results.
Any evaluation program should have the following components with each numbered element below being a necessary part of the package:
- Independent Evaluator. A Independent Evaluator that is organizationally separate from the program administrator should be established. This entity, not the Program Administrator, should be the organization that determines if, when and which evaluations should occur. The Independent Evaluator should procure evaluation studies instead of the Program Administrator.
- Long-term and Secure Funding for the Independent Evaluator. The Independent Evaluator should have long-term funding, e.g., 5 years, that cannot be reduced, modified or redirected by the Program Administrator.
- Complete Access to Data. The Independent Evaluator should have access to all data at the same time as the Program Administrator. It should not have to depend on the Program Administrator to obtain raw data, analyses, reports, etc.
- Integration of Evaluation with Program Design and Implementation. Evaluation should be integrated into the design and implementation of programs so that data that is generated for program administration can also be used for evaluation purposes. This will reduce the time needed to conduct evaluations and will provide more timely assessments and feedback.
- Continual Evaluation. Evaluation should, where possible, be conducted on a continual basis instead of in a batch mode. The batch model of evaluation – let a program run for a year or more and then conduct an evaluation that takes a year or so – does not result in timely and effective evaluations. Program participants should be required to provide data on an ongoing basis that is needed for evaluation purposes even after their initial participation to inform existing and future programs.
- State-of-the-Art Evaluation Methods. Evaluations should use, where possible, state-of-the-art evaluation methods such as controlled experiments and quasi-controlled experiments.
- Public Reporting. The Evaluation Entity should be required to make routine, unfettered public reports.
Energy efficiency, if done right, is an important component of any clean energy effort, and getting it right requires objective, independent and timely evaluations.