Mark Stoelinga, Senior Scientist
Matthew Hendrickson, Manager of Energy Assessment
This article presents the executive summary of the study.
Vaisala has conducted a significant validation study of its due diligence wind energy assessment methodology. The study was based on 127 years of energy from commercial wind farm operations at 30 different wind farms in the United States, Europe, and Asia. Pre-construction assessments were performed with Vaisala’s current methodology in a blind retrospective forecast framework. The mean bias error for all wind farm years (where “error” is defined as actual energy produced minus the pre-construction long-term estimate, expressed as a percent) was +0.1%, with a 95% confidence interval of ±4.3%. The standard deviation of the 1-year errors was 8.8%, somewhat lower than Vaisala’s mean estimated 1-year uncertainty on energy of 10.6%, indicating that Vaisala’s estimated uncertainties are somewhat conservative.
Figure 1. Map showing the approximate regional distribution of wind farms included in the study. The number inside each marker specifies the number of wind farms in the region indicated.
In this study, Vaisala has also examined whether there is a relationship between prediction error and pre-construction estimate uncertainty. To do this, a Monte Carlo model was developed that enabled simulation of thousands of validation studies similar to the one actually carried out. A simple metric for quantifying the uncertainty/error relationship was defined, which is the slope of the best-fit line on a scatter plot of absolute wind farm year errors versus 1-year uncertainty estimates. The slope in Vaisala’s study was 0.57, and this slope implied a median “uncertainty model skill factor” (i.e. the fraction of variance in the uncertainty explained by the uncertainty model, with 0.0 showing no skill and 1.00 showing perfect skill) of 0.51.
In other words, Vaisala’s result was the median result expected by a “half-perfect” uncertainty model. However, given the sample size of the current study, the 95% confidence lower bound was a skill factor of only 0.09 (i.e. Vaisala is 95% confident that its uncertainty model is at least 9% perfect). These results illustrate how difficult it is to confirm a strong positive uncertainty/error relationship. Even with 1000 additional wind farm years and the current results still standing, Vaisala would still only be able to make a statement with 95% confidence that we have an uncertainty model that is at least 25% perfect.
This study is good evidence that Vaisala’s wind energy assessment process is calibrated to actual production while showing that there is at least some positive relationship between uncertainty and prediction error.
For the full 20-page paper, register here: https://goo.gl/Nr3M3k
Filed Under: Uncategorized
