Related Expertise: Power and Utilities
Is Electricity Pricing Running Out of Gas?
To achieve net-zero emissions by 2050, electricity systems around the world need to use far more variable renewable energy (VRE) generation, particularly solar and wind power. The transition from fossil fuels to VRE is unlikely to be smooth. As previous articles in this series have shown, VRE creates physical challenges due to its intermittency and variability, with different forms of energy storage emerging as critical solutions. But VRE also creates market challenges for traditional wholesale energy-only markets, with storage unlikely to provide the solution.
Natural gas has a key role to play in setting prices in today’s wholesale energy-only electricity markets. Because of the way prices are set in these markets, gas prices disproportionately determine the price of electricity. As the entry of substantial VRE generation and storage causes coal-fired generators—and subsequently gas-fired generators—to close, it has the potential to undermine this mechanism. Left unchecked, this could lead to high and unstable electricity prices, impairing vital investment signals.
Given the importance of renewables in meeting future global energy needs, are the days of wholesale energy-only electricity markets numbered? Not necessarily. Depending on the dominant technology that emerges to manage the variability of renewable energy and to ensure that adequate supply exists—whether it be zero-emissions gas, energy storage, or flexible demand—market designers and companies can adapt to the new environment, by adjusting the market design or changing their behavior. However, the wholesale energy-only market may play a very different role from the one it does today.
This article is the fourth in a series exploring changes to our electricity and broader energy systems. (See “Other Articles in the Series.”) In this article, we explore how higher penetration of variable renewable energy will impact the way electricity is priced in wholesale energy-only markets and the potential implications for electricity market design.
Other Articles in the Series
This article is the fourth in a series exploring changes to our electricity and broader energy systems—and the implications of those changes for consumers and governments—that will arise from the massive shift to electrification and variable renewable energy needed to achieve our net-zero emission ambitions. In the first three articles, “Why Your Company Needs to Be an Electricity Trader,” “Rising to the Challenges of Integrating Solar and Wind at Scale,” and “Will Electricity Be Free? Not When You Really Need It,” we examined how the advent of variable renewable energy creates both physical stability and market volatility challenges for our electricity systems. We covered the potential solutions to key technical challenges, the implications of greater volatility for governments and companies, and the opportunity for electricity-intensive users to create advantage by being more flexible in the way they consume electricity.
How Electricity Pricing Works
Starting in the late 1990s, a swath of countries opened their electricity markets to competition to increase efficiency and reduce the cost of electricity. As part of this liberalization, market operators introduced wholesale energy-only markets that permitted different types of power generation plants to be dispatched efficiently and were intended to be the primary signal for new investment. Many different designs for liberalized electricity markets can arrive at similar economic results. As an example, the Australian National Electricity Market (NEM) wholesale energy-only market operates as follows:
- Generation plants bid the minimum price at which they are willing to generate electricity. They do this separately for every period (in Australia this happens every five minutes). In theory, these bids reflect generators’ short-run marginal cost (SRMC), the cost of producing an additional unit of power, which in turn is determined largely by their fuel costs.
- Plants are dispatched in a “merit order,” with the plants offering the lowest bids dispatched first, followed by those with higher bids, until electricity generation is sufficient to meet demand.
- The price of electricity at any given time is set by the highest-bidding generator that, taken together with all other generators in the merit order, must be dispatched to meet demand. Market operators pay this price to all generators that have been dispatched, meaning that some plants receive a price that exceeds their SRMC, thus providing a return on their capital investment.
- When demand is so high that generators can’t meet demand, prices reach a very high level, known as the value of lost load. In Australia, this level is $15,500/MWh—which is up to 200 times higher than typical prices. In other markets, a market price cap is set much lower (in Europe, the cap is €3,000/MWh).
- In many jurisdictions, companies use financial derivative markets and hedging contracts—often of quite limited length and liquidity—to manage their exposure to spot-market variability.
- Prices (and forecasts of future prices in this market) are intended to provide the bulk of the investment signal to plant owners and investors. In some markets, other mechanisms such as capacity payments provide supplemental signals.
Aside from SRMCs, other factors impact generators’ bidding strategies. These include whether there is a limit on the total amount of electricity they can generate (that is, whether they face an energy constraint) and how easily their output can be adjusted. On the basis of these factors, we segment generators into five categories. (See “Electricity Generators Have Different Roles in Price Setting.”)
Electricity Generators Have Different Roles in Price Setting
We have identified five categories of generators, based on whether there is a limit on the total amount of electricity they can generate (that is, whether they face an energy constraint) and on how easily their output can be adjusted (their controllability).
Price Setters. These plants are easy to control and generate power for a sustained period. They bid to supply electricity on the basis of their own SRMC, which they determine by means of an external reference price: the underlying cost of fuel, which for most price setters is natural gas.
Price Opportunists. These plants are easy to control, but unlike price setters they are energy constrained: they can produce only a limited supply of electricity before their stored energy runs out. Examples include batteries, conventional hydro plants, and pumped hydro plants (which generate power as water moves from a higher to a lower reservoir and have an SRMC close to zero). They bid on the basis of the opportunity cost of deploying their energy store at another time (we call this opportunistic pricing), and they adopt bidding strategies designed to optimize the value of this limited supply. To do so, they seek to generate electricity primarily during peak price periods. Usually, they bid to undercut other generators—often price setter technologies—rather than bidding on the basis of their own SRMC.
Price Followers. These plants, which include black-coal-fired generators, have a low SRMC and so tend to be among the first to be dispatched. Their controllability is limited, but they can generate power for a long time. Although they typically bid at least part of their generating capacity on the basis of their SRMC, they often bid a share of their capacity to undercut the price setters rather than biding at their SRMC. This shadow bidding strategy means that they earn more revenue when they are the marginal generator.
Price Takers. These generators are most commonly variable renewable generators, such as wind and solar plants. Because they have a near-zero SRMC, they typically have to accept the market price set by other generators farther along the merit order. They set the electricity price when wind or sun is abundant, but that price is only ever at or near zero in such cases, due to their very low SRMC. Whereas dispatchable power plants can readily ramp up or down to meet demand, the only time it makes sense for price takers to curtail generation is when prices turn negative.
Price Absorbers. Nuclear power plants and brown-coal-fired generators play a negligible part in setting the price of electricity. It’s expensive and technically challenging for these plants to alter their output, so they accept the price that other generators set. They set the price only during extreme market conditions. For example, inflexible power plants sometimes bid at large negative prices to secure buyers for their energy because they cannot stop generating.
Price Setters. These plants are easy to control and generate power for a sustained period. They bid to supply electricity on the basis of their own SRMC, which they determine by means of an external reference price: the underlying cost of fuel, which for most price setters is natural gas.
Price Opportunists. These plants are easy to control, but unlike price setters they are energy constrained: they can produce only a limited supply of electricity before their stored energy runs out. Examples include batteries, conventional hydro plants, and pumped hydro plants (which generate power as water moves from a higher to a lower reservoir and have an SRMC close to zero). They bid on the basis of the opportunity cost of deploying their energy store at another time (we call this opportunistic pricing), and they adopt bidding strategies designed to optimize the value of this limited supply. To do so, they seek to generate electricity primarily during peak price periods. Usually, they bid to undercut other generators—often price setter technologies—rather than bidding on the basis of their own SRMC.
Price Followers. These plants, which include black-coal-fired generators, have a low SRMC and so tend to be among the first to be dispatched. Their controllability is limited, but they can generate power for a long time. Although they typically bid at least part of their generating capacity on the basis of their SRMC, they often bid a share of their capacity to undercut the price setters rather than biding at their SRMC. This shadow bidding strategy means that they earn more revenue when they are the marginal generator.
Price Takers. These generators are most commonly variable renewable generators, such as wind and solar plants. Because they have a near-zero SRMC, they typically have to accept the market price set by other generators farther along the merit order. They set the electricity price when wind or sun is abundant, but that price is only ever at or near zero in such cases, due to their very low SRMC. Whereas dispatchable power plants can readily ramp up or down to meet demand, the only time it makes sense for price takers to curtail generation is when prices turn negative.
Price Absorbers. Nuclear power plants and brown-coal-fired generators play a negligible part in setting the price of electricity. It’s expensive and technically challenging for these plants to alter their output, so they accept the price that other generators set. They set the price only during extreme market conditions. For example, inflexible power plants sometimes bid at large negative prices to secure buyers for their energy because they cannot stop generating.
Typically, gas-fired mid-merit generators (which adjust their output to match demand throughout the day) and peaking generators (which run at times of high demand) set the wholesale spot price of electricity most of the time, either directly or indirectly, so we call them price setters. (See Exhibit 1.) The name is apt not only because they set the price directly when they are the marginal generator (that is, are last in the merit order), but also because they set the price indirectly via the bidding strategies of other generators that use price setters’ bids as a reference price.
We call generators such as hydro generators and energy storage, which have constraints on the amount of energy they can produce, price opportunists because they base their bids on the opportunity cost of deploying that energy at another time. Price opportunists’ supply bids are intended to maximize revenue from the limited amount of electricity they can generate and are typically set just below those of price setters. Consequently, their bids are often impacted by the price setters’ fuel costs.
We call generators with limited flexibility and no energy constraint price followers. These generators typically have a low SRMC. In most instances, price followers bid on the basis of their SRMC. On other occasions, their bids reference the SRMC of the next-highest price-setting technology to earn more revenue when they are the marginal generator—a strategy known in Australia as shadow bidding. Such bidding strategies extend the influence of gas prices on the price of electricity.
The Importance of Natural Gas
The bidding strategies of different types of generators ensure that price setters and the natural gas that powers them have a disproportionate role in determining the price of electricity. The consequence can be seen in Australia: although gas-fired generators contributed only 12% of power generation from 2014 to 2021, they directly or indirectly impacted the electricity price for nearly 75% of total power generation.
Gas-fired generation also plays a dominant price-setting role in the European Union, largely as the marginal generator. There, gas-fired generators represent less than 20% of total generation but set the price more than two-thirds of the time.
As a result of this effect, gas prices materially affect electricity prices, with movements in gas prices leading to significant changes in electricity prices. For example, from 2012 to 2020, the average monthly wholesale spot price in Australia increased by $8/MWh for every $1/GJ rise in the price of gas.
This relationship also means that crises in gas markets, such as the current global natural gas market crisis, spill over into electricity markets. For example, from May 2022 to July 2022, Australian domestic gas prices reached $40/GJ, a four- to five-fold increase over typical prices, and this increase flowed through to electricity prices at a similar multiple of typical prices.
The Price-Setting Role of Gas Persists, Even as VRE Enters the Market
As VRE generation increases, the amount of electricity that gas-fired generators produce declines. It might be tempting to think that greater VRE generation would also reduce the role that gas plays in setting the price of electricity. But its price-setting role persists.
In Australia, the amount of electricity that gas-fired generators produced fell from 15% in 2014 to just 8% in 2021 due to increased electricity supply from VRE sources. Over the same period, the role of gas as the price setter did diminish, although by a comparatively smaller amount—from 70% of the time in 2014 to 50% in 2021. However, this change hides an underlying cycle of events and underplays the persistent role of gas.
At first, as VRE enters the market, the influence of gas as a price setter declines. This is because competition from VRE sources causes coal-fired generators to more frequently bid at their SRMC rather than shadow-bidding on the basis of gas-fired generators’ bids. Subsequently, however, mid-merit coal-fired generators exit the market because they lack the flexibility to operate during long periods when the market price is below their SRMC, and this causes gas to resume its dominant price-setting role. This cycle occurred in Australia in 2017 and is likely to recur in 2023 as additional coal-fired generating capacity closes. The result is that gas remains highly influential in setting the electricity price in Australia, despite commanding a reduced share of generation.
The Only Thing Worse than Gas Setting the Electricity Price Is Gas Not Setting the Price
Over the long term, the natural-gas-fired peaking generators that remain in operation hinder the goal of a zero-emissions electricity system. As a result, without zero-emissions fuels or abatement technologies, they would be forced to shut. The withdrawal of these generators, as well as of emissions-intensive price followers, would mean that the only generators left to set the electricity price would be price opportunists, such as storage.
As we mentioned earlier, because price opportunists are energy constrained, they look for the best pricing opportunity at any given time. Consequently, they tend to submit supply bids that take advantage of the high prices that price setters secure at times of peak demand. But if the price setters were to exit the market, price opportunists would have no yardstick for deciding the level of their own bids. Electricity price formation would effectively be decoupled from its key external reference price—the price of gas—causing electricity prices to fluctuate between extremes. This, in turn, would make financial derivatives and hedging contracts extremely expensive, as markets struggle to price in this level of risk. The spot market would no longer be the main driver underpinning investment decisions.
Recent events in Australia show how this situation could play out. In late June 2022, the Australian electricity market became energy constrained. First coal-fired generators (due to outages, flooding at coal mines, and supply chain issues) and then gas-fired generators (due to increased gas demand and lower storage levels) became fuel constrained. As a result, the economics suggests that these generators tended to behave more like price opportunists, generating electricity at the most valuable times. This would imply that electricity prices might be limited not by gas prices but instead by a price-capping mechanism known as the cumulative price threshold. For example, in the week prior to July 17, when gas supply constraints were most challenging, the demand-weighted average electricity price across the mainland NEM states was $500/MWh—above the SRMC of peaking generators.
The Effectiveness of Wholesale Energy-Only Markets Will Depend on Technological Developments
What role will wholesale energy-only market pricing play in the future? How must electricity markets evolve to produce clear price signals and drive effective investment decisions? We believe that the answers to these questions depend on which of three types of technology emerge as the dominant, physical firming solutions (see Exhibit 3):
- Zero-emissions gas-fired generation
- Greater flexibility of demand, supported by new processes and technologies
- Energy storage and other technologies that time-shift supply or demand
External factors such as technology costs, local resources, and the makeup of local industry will determine which technologies are the dominant firming solutions in different markets around the world.
The implications for market players depend on whether the dominant solutions are energy constrained, as in the case of storage, or energy unconstrained, as in the case of gas-fired generators, and whether the solution is on the supply side or the demand side. (See Exhibit 4.)
Zero-Emissions Gas: Wholesale Energy-Only Markets Continue to Drive Investment
Generation powered by zero-emission gases such as green hydrogen, biogas, or natural gas with carbon capture, utilization, and storage (CCUS) technology could replace today’s natural-gas-fired generators.
In this scenario, sufficient price setter generating capacity could exist to enable price setters to set electricity prices in the spot market, and wholesale energy-only markets would continue to play a meaningful role in driving investment, as they do today.
Greater Flexibility of Demand: Wholesale Energy-Only Markets Evolve to Ration Demand to Meet Supply, and They Continue to Drive Investment
Energy users with highly flexible demand could play a crucial role in setting the price of electricity. Until now, end-users have not participated directly in the wholesale energy-only market at scale. Instead, they have used out-of-market demand response mechanisms: reducing their consumption during extreme demand periods.
However, some energy users share important characteristics with today’s price setter generators that make them effective replacements. They have controllable production processes and can vary their output over extended periods. In other words, their electricity demand is not time critical. Most importantly, they can base their production decisions on an external reference price—the revenue they earn by selling their product weighed against the variable cost of the electricity and other inputs needed to produce it.
Some of these users, such as hydrogen producers and crypto miners, can treat production as entirely optional. Others, such as steelmakers using electric arc furnaces (EAFs) and smelters, may have some degree of flexibility in the total volume they produce and can use stockpiles to create flexibility in their production schedules.
This flexibility puts them on a different level from users that can flex their electricity demand for short periods but must meet an external constraint over a longer time period (time-critical demand). Users that fall into the latter category include electric vehicles (EVs), which must be charged to run, and water networks that process and pump water.
To set the price of electricity, highly flexible users such as hydrogen producers and steelmakers would conduct a two-step process:
- Bid to purchase electricity when it is priced below a certain level determined by the external reference price of their end-product, enabling them to make a profit contribution.
- Curtail production if the electricity price rises above that level, rather than continuing to produce at a loss. This would be a significant change from industrial practices today, which see energy users focus on maximizing output.
Significant volumes of price-setting demand could either replace or supplement the role of gas-fired generators in wholesale energy-only markets and drive investment decisions. Rather than rationing flexible supply to meet fixed demand, the wholesale energy-only market would ration flexible demand to meet the variable supply of renewable energy.
Will there be enough energy users possessing these characteristics to make a difference? There certainly could be in markets that produce significant volumes of hydrogen for export or that have large energy-intensive manufacturing industries.
Energy Storage and Other Time-Shifting Technologies: Contracting Markets Drive Investment
Energy storage technologies are emerging as a promising solution for the physical challenges posed by a high penetration of VRE because they firm VRE supply to meet demand. But they do not solve the market challenges.
Because storage and similar solutions are price opportunists, wholesale energy-only markets would create periods of unacceptably high volatility for both consumers and investors. This would also be the case if flexible demand with price opportunist characteristics (such as EVs with a time critical electricity requirement) were a major firming technology.
Electricity markets would have to undergo a fundamental redesign so that longer-term contract markets, such as capacity markets, or separate markets for variable power (with generation “as available”) and firm power (with generation “on demand”) drove investment decisions. Under this approach, remuneration in the variable power market would be based on contracts for difference, with prices reflecting the long-run marginal cost of existing generators. Emerging market redesign proposals from the governments of Greece and the UK are beginning to countenance such a split.
The wholesale energy-only market would have a limited investment role. Its primary raison d’être would be to signal the efficient dispatch of different types of generators, storage, and flexible demand via a merit order. If this redesign doesn’t occur, companies will need to contract outside of the market—for example, by increasing vertical integration or by entering into long-term contracts for energy and capacity.
As the world moves toward greater VRE generation, liberalized electricity markets face a crisis of purpose. Up to now, thermal generators—particularly natural gas-fired generators—have had a dual function, providing both physical stability and market stability to our electricity systems. But while electricity systems may be able to function physically without natural gas, electricity market pricing will break down if other solutions don’t replace the role that gas plays in setting wholesale spot market prices for electricity. Markets will need to be fundamentally redesigned to adapt to this new environment.
How we best adapt our electricity markets in the long term depends on which types of technology—zero-emissions gas, highly flexible demand, or energy storage—become the dominant firming solutions. As such, market designers can’t make decisions in isolation from technology. Moreover, if they are to make net-zero electricity systems a reality, they must take care that their reforms go beyond solving today’s issues and ensure that their designs are compatible with likely technological developments.
ABOUT BOSTON CONSULTING GROUP
Boston Consulting Group partners with leaders in business and society to tackle their most important challenges and capture their greatest opportunities. BCG was the pioneer in business strategy when it was founded in 1963. Today, we work closely with clients to embrace a transformational approach aimed at benefiting all stakeholders—empowering organizations to grow, build sustainable competitive advantage, and drive positive societal impact.
Our diverse, global teams bring deep industry and functional expertise and a range of perspectives that question the status quo and spark change. BCG delivers solutions through leading-edge management consulting, technology and design, and corporate and digital ventures. We work in a uniquely collaborative model across the firm and throughout all levels of the client organization, fueled by the goal of helping our clients thrive and enabling them to make the world a better place.
© Boston Consulting Group 2024. All rights reserved.
For information or permission to reprint, please contact BCG at permissions@bcg.com. To find the latest BCG content and register to receive e-alerts on this topic or others, please visit bcg.com. Follow Boston Consulting Group on Facebook and X (formerly Twitter).