Predictions are Hard, Particularly about the Future
This blog post is by necessity an oversimplification, and it is for general information and entertainment purposes only. It is not business, legal, or investment advice. Content remains subject to change.
News articles about energy forecasts can be confusing, given vastly different outlooks on where the world is headed. And as of late, even the International Energy Agency (IEA) has been caught in a political storm, with Opec calling the IEA’s peak fossil fuel forecast “dangerous” and others claiming that the IEA is no longer credible. So I dug out IEA World Energy Outlook reports from 10 and 20 years ago and compared them to what actually happened so far. Some of the differences were just astonishing!
Firstly though, the word “forecast” is not quite accurate. The 2002 WEO still had only one “forecast” number for each variable, but by 2012 the approach had shifted to scenarios.:
2012 WEO CPS: Current Policies Scenario, assuming no change to policies
2012 WEO NPS: New Policy Scenario, taking into account existing commitments
2012 WEO 450: 450 Scenario, limiting warming to 2C
Thinking in terms of scenarios instead of “forecasts” should remove the vast majority of political arguments over whether the “forecasts” are credible or not. Scenarios help to understand what drives a system and how it can be steered toward more desirable outcomes. To quote George Box, “All Models are wrong, but some are useful.”
That said, scenarios typically provide high and low cases. If actual outcomes are outside these “bookends” then usually something was amiss. Especially when the outlook horizon was relatively short, e.g. 2020 for the 2012 WEO. And at least three variables qualify as significant outliers.
Here we go!
“About right”
WEO Scenarios indexed to Actual (see footnotes for details)
Oil Demand: A bit above the 2012 scenarios but below the 2002 outlook.
Natural Gas Demand: Similar to oil demand, above 2012 scenarios but below 2002 outlook, overall in the right ballpark.
Coal Demand: A lot higher than forecast in 2012, but within the 2012 forecast range. The main driver for differences vs the 2002 outlook appears to be China’s economic growth.
Electricity Generation: Above 2002 outlook but close to the upper end of 2012 scenarios.
“Wow”
WEO Scenarios indexed to Actual (see footnotes for details)
Oil Prices: The 2002 report talks quite confidently how oil prices will gradually rise from USD 21/bbl in 2011 to USD 29/bbl in 2030. That’s in real terms for the year 2000, i.e. increases to USD 42/bbl nominal when CPI-adjusted. The 2012 report in turn shows USD 112/bbl as the lowest-price scenario for 2020 - USD 147/bbl nominal when CPI adjusted as low-case! The 2022 actual average price was USD 98/bbl. As one of my professors used to say, “oil price forecasting is simple, take today’s price and put a funnel around it, then talk with confidence”.
Solar Energy: Simply astonishing. Actual global solar PV power production in 2022 was 2.2 times higher than the 2012 high case prediction for 2020, and 30 (!) times higher than the corresponding 2002 report forecast. Clearly no one foresaw the continuing steep decline in PV costs. Another clear case of “if it’s profitable, it scales”!
Electric Vehicles: Not a technical error in the above chart: EVs were not featured in the 2002 report, and with a high-case new light vehicle market share of 4% in the 2012 report for 2035, near-zero for 2020. Reality was 14% market share in 2022, i.e. 3.5 times above the 2012 high-case forecast and 12 years earlier. With EV adoption driving most of near-term oil demand uncertainty, this remains worth watching.
Takeaways
Usefulness: Have the WEO’s provided useful models? Yes, and with much more transparency than most other energy models / scenarios. Kudos to the IEA for avoiding “black box” methodologies found in so many other reports. If nothing else, the reports are great textbooks about what drives the global energy systems.
Disruptive Technologies: Even in systems as vast and capital-intensive as energy, disruptions are possible within relatively short time horizons. Both solar PV and electric vehicles were well known technologies in 2012, but the learning- and consequently cost- and adoption-curves were still vastly underestimated by even the most knowledgeable researchers in the field. Even the shale revolution was still underestimated in 2012. Gas-to-Liquids was seen as a promising growth area. Thus with record amounts of funding for new energy technologies from solid state batteries to nuclear fusion to thermal energy storage, there is a high chance that the 2050 energy landscape will look vastly different from today’s.
Energy Prices: The fact that oil prices developed so differently from what was expected in 2002 and 2012 means that instead of “forecasting” commodity prices, anyone exposed to them should be ready to operate in scenarios that could be vastly different from the status quo. It’s about robustness more than it is about fine-tuning and optimization. Oil & Gas producers have largely learned the lesson and shored up balance sheets accordingly.
Reading the old reports was a great walk down memory lane. Better than most books, highly recommended for long winter evenings!
Next time - Strong Convictions Loosely Held: Residential Solar PV in Alberta.
Footnotes - More details on the approach
This blog looks back at the IEA’s 2002 and 2012 World Energy Outlook (WEO) reports, with a comparison of scenarios to what actually happened as outlined in the 2021 and 2023 WEO reports.
Summarizing these reports in an easily-digestible format is challenging at best, thus this blog post suffers from the same flaw as other news articles - oversimplification. The reports are very detailed and long (2002 - 533 pages, 2012 - 690 pages, 2023 - 355 pages), yet surprisingly enjoyable to read (compared to e.g. the IPCC reports). The modeling is overall bottom-up, whereas many other forecasters model top-down e.g. through GDP-linked metrics. This doesn’t necessarily mean higher precision, but it leads to a lot of insights for the reader. And one thing the IEA does exceptionally well is transparency of modeling assumptions.
A few important details:
The 2002 WEO report makes forecasts for 2010, 2020, 2030. The 2012 reports references 2020, 2025, 2030, 2035. And the 2023 report forecasts for 2030 and 2050. Actual numbers for 2020 were distorted due to Covid, with 2022 being the closest to “normal” recovery situation. I thus took the liberty of comparing 2022 actuals to 2020 predictions for the commodity demand, electricity generation and oil price variables. Differences should be immaterial for the very high level conclusions drawn here. For EV adoption the scenarios outlined near 0% adoption through the 2020s anyway, thus choice of “actual” year did not matter materially either. For solar PV generation the “actual” figure refers to 2020, as that figure did not seem particularly distorted by Covid-19.
Oil price CPI adjustment: the prices quoted for WEO 2002 are in real terms 2000, WEO 2012 forecasts are in real terms 2011, and those for WEO 2023 in real terms 2022. To be directly comparable, the forecasts thus need to be adjusted for inflation between reference years. I have done this using the US Bureau of Labor Statistics CPI calculator for January 2002/2012 vs December 2022.
Here is the input data for the above graphs and discussions:
This blog post remains for general information and entertainment purposes only; it’s not a scientific study or investment advice. As such I hope that despite all its flaws and oversimplifications it provides some useful insights.