Replication of the publicly-released 2021 Sandia PVPMC dataset

Executive summary

• What was tested: Heliosolve replicated two scenarios from the publicly-released 2021 Sandia PV Performance Modeling Collaborative dataset — a Panasonic HIT panel and a Canadian Solar mono-crystalline panel, both fixed-tilt, both at the Sandia research site in Albuquerque.
• What we found: Modelled DC power matched the measured reference to −1.93% (Panasonic HIT) and +0.52% (Canadian Solar) on the year, with measurement-vs-model correlation of R² = 0.9843 and 0.9862. Both biases sit inside the 2021 PVPMC blind intercomparison cohort’s median band (−3.3% annual DC energy across 29 submissions).
• Why it matters: These results address the modelling share of the industry’s ~8% yield-forecast gap. Operational, financial and degradation contributors lie outside this report. Tighter modelling on the DC side supports lender P50/P90 inputs as one factor among several.

Scope: two scenarios at one research site, one year of measured data. Self-conducted post-release replication (Heliosolve did not submit to the 2021 blind round); not a third-party audit.

Why this matters

Modern utility-scale solar plants produce roughly 8% less energy on a weather-adjusted basis than their pre-construction forecasts had predicted. The gap has multiple drivers — soiling, availability, degradation, EPC quality, weather inputs, and modelling. This report addresses the modelling share.

• The 8% shortfall is real and measured: in its 2025 update, kWh Analytics reports an 8.6% mean shortfall across 34,000 system-months of operating data.
• It feeds straight into project finance: higher modelling uncertainty widens the gap between best-case and worst-case revenue forecasts, which can depress lender debt-service-coverage margins.
• Simulator-vs-measurement gaps are documented in the literature: a 9-plant close test by NREL of one reference simulator reports annual-energy prediction errors below 3% for fixed-tilt installations and below 8% for tracker installations; the 2021 PVPMC blind intercomparison cohort itself reported a median annual DC-energy bias of −3.3% across 29 submissions.

What we validated

The validation reference is the publicly-released 2021 Sandia PV Performance Modeling Collaborative (PVPMC) dataset — the same dataset that 29 blind-cohort participants from 28 institutions in 12 countries used to test their simulators. The full measurement record was made openly available in April 2023 by Sandia National Laboratories. Heliosolve did not submit to the 2021 blind round; the results presented here are a self-conducted post-release replication.

Why this is the right benchmark

• Fixed, public ground truth: Heliosolve’s local input data was checked against the public Sandia release row by row. The result: zero mismatches across 26,280 measurement cells per scenario (front-side sunlight, panel temperature, DC power) to two decimal places. This means simulation tuning to measurement after the fact would be detectable against the cohort’s published error band.
• Two technologies, same site: the replicated cases cover two distinct solar-panel technologies (Panasonic HIT and Canadian Solar mono-crystalline) on the same fixed-tilt rack at the Sandia Albuquerque research site — a clean test of whether the simulation chain works across different cell types under identical weather inputs.
• Peer-reviewed methodology: all metrics follow the methodology published by the PVPMC in Theristis et al. (2023).

The result

Over a year of measured data on both Albuquerque scenarios, Heliosolve’s modelled DC power matched the measured reference to −1.93% (Panasonic HIT) and +0.52% (Canadian Solar mono-crystalline) — both well inside the 2021 PVPMC blind intercomparison cohort’s own published error band (−3.3% median annual DC-energy bias).

The chain holds together at each step: sunlight on the panel reproduced to −0.20%, panel temperature to +0.97%, and DC power to −1.93%. The simulation is therefore well-calibrated at the level of each physical sub-model, not just at the aggregate DC output — the agreement does not depend on errors cancelling out.

What this changes for project finance

Tighter simulation on the DC side addresses one element of the forecast-uncertainty budget that lenders use to size debt and equity returns. Pacudan (2016) documents the link between resource-estimation uncertainty and project-finance metrics — the narrower the modelled distribution of plant output, the better the debt-service-coverage ratio and net present value of the asset.

Scope, limitations, and the next step

This validation covers two scenarios at one research site (the Sandia Albuquerque site) over a twelve-month window drawn from the publicly-released 2021 PVPMC dataset. Before the engine can be considered generally validated for production-grade forecasting, the same exercise needs to be repeated on additional sites, additional years, and additional cell technologies. That is the next phase of work.

What was filtered out

The metrics in this report are computed on the hours that pass the standard validation filters published with the Sandia dataset: front-side sunlight ≥ 100 W/m², DC power ≥ 50 W, solar elevation above 5°, and ambient temperature between −5 °C and +45 °C. Night-time hours and sensor-outage rows (encoded as zero values) are also excluded. After filtering, ≈ 3,569 hours remain for the Panasonic HIT scenario and ≈ 3,591 for the Canadian Solar scenario, out of the year’s 8,760 hours.

Glossary

Sunlight on the panel surface (POA)

The actual sunlight reaching the panel’s working face after accounting for the panel’s tilt, the sun’s position, and any shading.

Panel temperature

The operating temperature of the cells inside a panel under sunlight, wind and ambient air. Higher temperature reduces electrical efficiency, so getting it right matters for DC-power output.

DC power

The electrical power produced by the panels before it reaches the inverter, which converts DC to grid-quality AC.

Annual bias

The difference between the simulated value and the measured reference, integrated over the validation window, expressed as a percentage of the measured total. A bias of −1.93% means the simulation came in 1.93% below the measured total over the year.

Correlation (R²)

How well the simulated and measured values agree on a 1:1 line. R² = 1 would be perfect; R² = 0.98 means the simulation explains 98% of the hour-by-hour variation in the measured signal.

nRMSE / MAPE

Average size of the hour-by-hour difference between simulation and measurement, with two different ways of expressing the average. Both are reported in percent of the measurement.

P50 / P90

Probabilistic forecast percentiles. P50 is the median (best-estimate) annual production; P90 is the level exceeded with 90% probability, used by lenders as a worst-case sizing metric.

References

1. Theristis, M., Riedel-Lyngskær, N., Stein, J. S., et al. (2023). Blind photovoltaic modeling intercomparison: A multidimensional data analysis and lessons learned. Progress in Photovoltaics: Research and Applications, 31(11), 1144–1157. DOI: 10.1002/pip.3729
2. Theristis, M., and Stein, J. S. (2023). The 2021 Blind PVPMC Modeling Intercomparison. Sandia National Laboratories Technical Report SAND2023-03135. OSTI 2293576. Accompanying open-access dataset on the DuraMAT Data Hub (DOI: 10.21948/1970772).
3. kWh Analytics (2025). 2025 Solar Risk Assessment: Market Insight & Empirical Performance Data.
4. Pacudan, R. (2016). On the link between resource-estimation uncertainty and project-finance metrics.

The complete reference list, the hour-by-hour scatter figures and the full methodology appendix are in the PDF.