In the years since, I have seen more and more of my neighbors turn into prosumers. I recently became one myself, with solar panels and a battery storage system installed in my house. I also drive an electric vehicle (EV). The distant future has arrived much sooner than I expected, at least in my neighborhood. And, while California continues to dominate the nation in the sheer number of prosumers and EVs, it is not difficult to imagine a not‐​so‐​distant future in which much of the nation will begin turning into Prosumer Land.

The Consumer Revolution

A revolution is underway in the electric utility industry. The signs of this were evident long before the Great Recession of 2008–2009 slowed load growth. I spoke at Goldman Sachs’ Annual Power Conference in New York City soon after the recession ended and made that point. But the facial expressions of the investment analysts in the room told me they were not buying it. I was invited to speak at the same event two years later. I gave a similar message, saw a few people nodding their heads, but I’ve yet to be invited back there to speak again.

In 2014, I spoke at a conference on the outlook for electricity sales and peak demand. My message of flattening demand resonated with the technical audience. Two of the three other panelists agreed with me. (The fourth insisted an industrial renaissance was underway that would propel growth.) The only issue among those who agreed with me was which forces were driving this change. Some said the primary force was utility demand‐​side management programs. Some said it was governmental codes and standards. Some said it was the arrival of distributed energy resources. And some said that it was fuel switching away from electricity.

Today, as we stand at the cusp of the third decade of the 21st century, the trend is no longer being questioned, probably not even at Goldman Sachs. Over the past decade, consumers have decisively and irreversibly changed the way they think about electricity, how they consume electricity, and when they consume electricity. And some have turned into prosumers.

Of course, as we have discovered, no two customers are alike. Even within the same household, husband and wife often differ on how they want to live their lives. Children introduce more uncertainty into the energy decision‐​making. Of course, all customers want choice, but they only want what they want. Yet, utilities often offer just one product to all customers in a “rate class” — delivered electricity at a certain rate — thereby avoiding accusations of discrimination. A few offer some choices, but these are often marketed in a jargon that would politely be called obscure and they use communication channels that sometimes don’t even reach the customer.

It’s safe to say that diversity is the hallmark of customer preferences for consuming electricity, just as it is for any other product or service. Electricity is no exception. Utility consumers fall into several categories. Some want bill stability and are willing to pay more for it. Some want the lowest bill and are willing to shift and reduce load. And some have gone organic in every aspect of their lives and want to buy only green power to mitigate climate change. Yet, most utilities simply offer a single rate to all of them. Imagine what would happen to sales at retailers like Nordstrom’s if they only sized their merchandise as “one size fits all.”

I recently called my local utility’s customer service number and asked which rate I should pick given that rooftop solar panels and battery storage were about to be installed in my house. I was told to pick such‐​and‐​such a rate as a starting point. My bill would now run 10 pages, but I should ignore all the pages except 1 and 3. I asked if the recommended rate would be the best rate for me since I also have an EV. She said there was no easy answer to that question. It would be best if I waited for another year to figure out my best rate, which of course meant that I may end up paying more in the next 12 months.

The Technology Revolution

Concomitantly with the revolution in consumer tastes, an all‐​embracing technological revolution is underway, spurred by the advent of digital technologies. Just about all customers have smart phones today. Currently, about half of all customers have smart meters. But smart price signals are only rarely being transmitted through those meters.

More and more customers have energy‐​efficient appliances with digital chips embedded in them. In fact, you can no longer buy energy‐​hogging appliances even if you want to. Some customers live in highly energy‐​efficient dwellings, some with solar panels on their roofs and even batteries for storage. In Hawaii, which has very high electric rates, some 60% of new solar installations in Honolulu are being paired with batteries. In California, where planned power shutdowns are being carried out to prevent wildfires, the same can be expected. This has temporarily pushed up storage battery prices, but they are on a long‐​term declining trend. Finally, more and more customers are buying or leasing EVs despite their high prices and short range, and despite their especially high prices in California and Hawaii.

Disintermediation of Utilities

Disintermediation of utilities involves the entry of third parties that sell products and services to utility customers that reduce utility sales and revenues. This trend is well underway and appears to be unstoppable. Utilities may think they are regulatorily protected monopolies, but customers keep divining creative ways to manage their energy use outside of utility (and commission) directives. This should not surprise anyone, but it does seem to have eluded more than one utility and one regulatory body.

Electricity consumers are going to act in their self‐​interest, just as they do in every other market. Their eyes glaze over when they are told they cannot do such‐​and‐​such because it would be an uneconomic bypass of the grid and create cross subsidies between customers.

Customers on the frontier of change want local control and grid independence. Consumer choice aggregation is taking off like never before in California and is being considered in several other states, such as Colorado and New Mexico. The drivers are many, ranging from consumer desires to consume green energy, have local control, and lower expenses. But the ultimate driver in most cases, as mentioned by a utility executive to me, is a deep‐​rooted anti‐​utility sentiment.

New entrants that are disintermediating utilities include global tech giants, start‐​ups with unwieldy names, and even home security firms and hardware stores. The electric customer is no longer the exclusive preserve of the regulated monopoly.

While talking to a senior officer of a large utility the other day, I mentioned the “prosumer” conversation I had in Florida a few years ago. I thought he would dismiss the scenario that the skeptic had laid out, much as I once did. Surprisingly, he said that he was finding himself more and more in that camp. He added that economic history tells us that no industry has remained a natural monopoly forever. Utilities must change their ways if they want to survive.

Armageddon?

At one time, the utilities conference circuit included talk of “death spirals” — utilities slowly collapsing financially as a result of market change. Today, the talk is of sudden “Armageddon.” Whether the end is at hand or a chimera won’t be known for another decade or two. Still, if utilities and regulators continue to do business as they have for the past century, they will accelerate the demise of the electric industry.

In a Harvard Business Review article entitled “Marketing Myopia,” marketing professor Ted Levitt wrote ominously:

Every major industry was once a growth industry. But some that are now riding a wave of growth enthusiasm are very much in the shadow of decline. Others that are thought of as seasoned growth industries have actually stopped growing. In every case, whenever growth is threatened, slowed or stopped is not because the market is saturated. It is because there has been a failure of management.

He specifically cited the example of railroads forgetting they were in the transportation business, not just the railroad business. He cautioned oil companies about the advent of electric vehicles and electric utilities about the advent of rooftop solar panels. What is noteworthy is that the article was written in 1960. It is even more relevant 60 years later.

Waiting for Godot

In the meantime, utilities and regulators are moving slowly — one might even say ponderously — through rate cases. Regulatory lag is breaking records, often running into years. The slowest‐​moving drama in history is being played out in hearing rooms from coast to coast, from ocean to ocean.

Consider these case studies from my career. I have observed these instances of delays and back‐​tracking first‐​hand:

1976 The Electric Power Research Institute (EPRI) initiated the Electric Utility Rate Design Study at the behest of the National Association of Regulatory Utility Commissioners on behalf of the industry. It was carried out over several years with the close involvement of commissions, utilities, academics, and consultants. Nearly a hundred reports were produced on various aspects of time‐​of‐​use (TOU) rates. The study got a major boost when Congress passed the Public Utility Regulatory Policies Act (PURPA) in 1978. The study came to two primary conclusions: First, it was cost‐​effective to deploy TOU rates — rates that fluctuate to reflect marginal prices during the electricity demand cycle. Second, TOU rates could be developed using either embedded costs, which was the tradition in the industry and the favorite of accounts, or marginal costs, which was the approach favored by economists. Luminaires such as Alfred Kahn, chair of the New York Public Service Commission, chaired the advisory committee in its first phase. I joined EPRI in 1979 and worked on the study for a year. The biggest barrier to the deployment of TOU rates back then was the lack of smart meters. Today 50% of homes have smart meters, yet less than 5% of homes have TOU rates. The biggest barrier has turned out to be political.

1980s This decade saw some limited deployment of TOU rates in certain states, but those efforts were soon eclipsed by the emergence of demand‐​side management to enhance economic efficiency and lower customer bills. The main policy instrument was financing and rebates. Pricing was judged to be the ideal policy instrument, but such policies were deferred for later consideration, once again because politics intervened. TOU rates were relegated to the world of academe. A cottage industry arose comprised of academics who designed and evaluated TOU pricing experiments.

1990s The industry began to move toward restructuring, inspired by the liberalization of power markets in Great Britain during the Margaret Thatcher era. Conferences were held on the next generation of pricing designs, which would factor in retail customer choice and market restructuring. Plenty of books, papers, and articles were published. Once again, academics and researchers thrived. Not customers.

2000s I was tasked with finding ways to enhance energy efficiency in the Kingdom of Saudi Arabia. I discovered that a major barrier was that prices for electricity were heavily subsidized. I started asking people if I could meet the person who set prices, but no one could tell me who that was or where he worked. The utility said it was probably the regulator. The regulator said it was probably the ministry. When I spoke to the ministry, officials there were evasive. I persisted. Finally, someone told me the King set the prices. I decided not to pursue the topic. I figured out that His majesty did not want to trigger a revolt on the Arab street by raising electric rates. He had raised the price of petrol a few years earlier, but that had triggered an adverse reaction, forcing him to roll back the prices.

2002 Around the time of California’s energy crisis, Puget Sound Energy, which serves the suburbs around Seattle, deployed very attenuated TOU rates (which it called “real‐​time pricing”). Customers saved hardly anything, and a revolt ensued when shadow bills were sent out showing that. The new CEO of the company, a long‐​time advocate of TOU pricing when he was at Pacific Gas & Electric, shut down the program. The utility could have improved the savings opportunities for customers by increasing the off‐​peak discounts but chose not to do so. The national movement toward TOU pricing was set back a decade. Regulators and utilities drew the wrong conclusion, that TOU pricing was to blame for the revolt, when the problem was with the specific design of the TOU rate and not with TOU pricing in general.

2002–2004 Soon after the worst energy crisis in its history roiled California’s power markets, several economists (including me) signed a manifesto that concluded in part that the best way to avoid another crisis was to reconnect the retail and wholesale markets that had become disjointed when the industry was restructured in 1998. In 2002, the California Public Utilities Commission initiated a proceeding on advanced metering, demand response, and dynamic pricing. An experiment, called the Statewide Pricing Pilot, was carried out jointly by the three investor‐​owned utilities in California to test the merits of dynamic pricing. It ran during 2003–2004 and was monitored through regular meetings of a stakeholder group. It showed conclusively that customers responded to dynamic pricing signals by reducing peak loads and shifting peak usage to off‐​peak usage. Within a few years, all three investor‐​owned utilities were given approval to move ahead with advanced meters. Their business cases included an ample dose of dynamic pricing. Two decades have passed, millions of dollars have been spent on a new crop of pilot programs to confirm (yet again) that Californians respond to changes in the price of electricity. So, almost two decades after the energy crisis, the state will witness the ultimate anti‐​climax: Very mildly differentiated TOU rates will be rolled out to all customers. No one will save much, even if they move all their load to off‐​peak hours. People will either ignore the rates or get annoyed. I see Puget Sound Energy, Part II, in the making.

2006 I was invited to speak on smart meters and smart rates by the National Association of Regulatory Utility Commissioners. In the years that followed, I was invited back nine times to speak on the same topic. After one of those sessions, a commissioner from New Jersey said she was impressed with the benefits of smart meters and wanted to know if there was some way to get those benefits without the meters. I wanted to tell her I wish there was a way to get the benefits of sunlight without the sun. But I bit my tongue and just smiled.

2007 The chair of the California Energy Commission noticed that only half of the goals the state had laid out for introducing price responsive demand in its Energy Action Plan had been achieved. She hired me to work with stakeholders to identify ways to enhance that percentage and reach the goal of having 5% of California’s peak demand be price responsive. My report recommended that the commission use its load management standards authority to require that all new homes be equipped with smart, communicating thermostats. This would allow critical peak pricing signals to be transmitted to central air conditioners, a major driver of peak loads, thereby balancing demand and supply in real time. Unfortunately, nothing came of the proposal after a conservative talk show host stirred up an Orwellian vision of the program for his radio audience.

2009 After speaking at a conference on demand response, I talked on the sidelines with the CEO of PJM, the grid system that serves much of the mid‐​Atlantic. I asked him if he liked the discussion of price responsive demand. He said he did not trust price response because it wasn’t tangible; it was not steel in the ground. His job depended on keeping the lights on. If the lights went out because the price response did not materialize, he would be out of a job. I responded that he couldn’t control the weather or the economy; he should be used to planning under uncertainty. Price response is not any more volatile than the economy or the weather, I noted, and he should be able to count on it. Besides, it would save consumers money. By the time I finished my point, he had turned away and was speaking with someone else.

2009 I carried out a study for the New York independent system operator on the benefits of real‐​time pricing. The quantified benefits were significant. But little subsequently happened because the issue fell under the dominion of the state commission, and it was reluctant to move on rate modernization because the state lacked smart meters. Of course, that was just a convenient excuse.

~ 2009 Inaction is not just a North American problem. About 10 years ago, in Saudi Arabia, I was presenting the final results of a project designed to promote energy efficiency in the country to the executive suite of the government‐​owned electric utility. Halfway into my remarks, a vice president asked me why I kept using the word “customer” over and over. His tone was testy. I was not sure what to make of his question because all the work I had done was designed to encourage customers to invest in higher‐​efficiency equipment. It could not have been a language problem because he spoke fluent English. I answered, “Because the customer is the king.” The audience’s faces blanched and I realized the gravity of what I had said. Mercifully, one audience member rescued me by saying that customers were writing letters to the editor complaining about the poor customer service of the utility.

2009 The Federal Energy Regulatory Commission conducted a state‐​by‐​state assessment of demand response potential and identified the best way to harness it was to deploy smart meters and offer smart rates to all customer classes. Several workshops were held with stakeholders and a national action plan was launched. But the idea failed on the launch pad because the implementation plan that followed was devoid of actionable policies, directives, and incentives. I wrote to the chair of FERC and said the plan was a damp squib. He asked if I knew the meaning of that British expression. What more was there to say?

2000–2010 Having observed the California energy crisis from afar, Ontario, Canada decided to roll out smart meters and deploy TOU rates as the default tariff in the mid‐​2000s. However, the price differential between the peak and off‐​peak periods was highly attenuated. Also, the TOU differential only applied to the generation portion of the tariff. Nonetheless, a three‐​year analysis carried out by a team of researchers (including me) showed that customers were reducing peak load by a few percentage points, but the savings were atrophying year after year. A recommendation that we had made in 2010 to accentuate the savings opportunities through dynamic pricing was ignored.

Late 2000s The Harvard Electricity Policy Group provides a good forum for discussing smart meters and smart rates. During one of my presentations at the event, a commissioner from Washington, DC asked me if customers would response to price changes, since electricity was a necessity. She asked me this question after I had shown an overwhelming amount of the evidence that customers do respond to price.

2010 At a major law school conference on the future of the utilities industry, I talked to the chair of the utilities commission about the delays in policymaking. He said that the utilities were frozen in time. Later, I made the same comment to a senior executive of the local utility. She said that the regulators were frozen in time.

2010s I have spoken a few times in Hawaii on smart grid and smart rates during the past decade. One of the state commissioners promised to write “a postcard to the future” to the mainland on how the state was going to become 100% renewable before 2050. Yet, to this day, the state has no smart meters, let alone smart rates. In the meantime, a third of single‐​family homes in Oahu have installed solar panels on their roofs. Some 60% of new solar customers are also installing batteries. I have seen several EVs on the road and Tesla has an incredible showroom right in the heart of Waikiki. Consumer have once again left the utility and the commission behind.

2011 After sharing the results of a dynamic pricing experiment with a senior utility executive, I recommended what I thought was the most forward‐​looking rate design from those that had been tested in the experiment. He picked an anodyne rate design. My face must have given away my inner thoughts because he added quickly: “I am not stopping you from writing your articles and giving your talks. But this is my company and I will do what I think is in the best interest of the company.”

2012 A workshop sponsored by the California Foundation on the Environment and the Economy reexamined the tenets of California’s inclining block rates. Three speakers — two professors from Berkeley and I — spoke at the event. This was followed by comments from several stakeholders. Following up on the workshop conclusions, the California Public Utilities Commission initiated proceedings to redesign the inclining block rates. Five steeply differentiated tiers had been created after the energy crisis. All the inflation that came in the years that followed was lumped onto the upper three tiers. After deliberating on the issue, the commission unanimously passed a rule to flatten the tiers. The five tiers would be replaced with just two. But at the last minute, to arrive at a unanimous decision, a super‐​user surcharge was introduced for large users. Currently, it stands at 55¢ cents per kilowatt hour for San Diego Gas & Electric and just under 50¢ for Pacific Gas & Electric. Simultaneously, the state wants to decarbonize completely by 2045 and it views electrification of buildings and transport as the best way to get there. But how do you convince consumers to switch to heat pumps when electricity is prohibitively expensive compared to natural gas? I have raised this issue with some of the energy division staff who are working on decarbonization. They said it’s an issue for the rate design group and they will get to it in the future. Once again, the can has been kicked down the road.

2012 I was retained by the Australia Energy Market Commission to examine the case for applying dynamic pricing for distribution tariffs. In Australia (as in Texas), customers have to choose a retail energy supplier. There is no default regulated supply option; the regulator only sets distribution tariffs. My final report recommended reforming this, but I was told there were political challenges to be overcome. We discussed a variety of different deployment mechanisms and ultimately devised a scheme that would make these rates mandatory for the largest customers, optional for vulnerable customers, and the default tariff for everyone else. I thought the recommendation was touched by Solomon’s wisdom. Alas, the government did not agree. To this day the recommendation has not been carried out.

2014 Minnesota initiated a process for creating the grid of the future. Demand response is a major priority of the state and studies indicate the best way to harness its potential is to deploy dynamic pricing to all mass‐​market customers. The state first began considering the deployment of smart meters and smart pricing in 2001, following the example of Puget Sound Energy. But the California electricity crisis prompted Minnesota to pull back. A pilot with various time‐​varying rates was scuttled. Finally, after years of deliberation, a simple TOU regime will be launched.

2015 I was invited by the New York Law School to be a keynote speaker at a conference on time‐​varying rates. The state energy czar opened the event, followed by the chair of the utilities commission. I gave my talk and hoped it would make a difference. To this day, the state is still trying to make up its mind about smart meters and doing pilots with innovative rate designs. New York’s energy vision is taking shape very, very slowly.

2019 While discussing rate reform in Texas, a former utility commissioner told me to wait another five years because the legislature had recently had a lot of turnover and the new lawmakers needed time to get up to speed. I said I have been hearing that for the past four decades.

2019 In a northwestern state, after I had testified for five hours spread over two days, a staff member walked me to my car and said, “Thanks for coming, but I think I the commission will just kick the can down the road.”

2019 In a Canadian province, I shared several ideas for moving customers to innovative rates to help utilities stay in step with their customers. I noted that there were EVs on the road there, just about everyone carried a smart phone, and consumers there were buying energy‐​efficient appliances. That’s why it was time to modernize rates. I was told the status quo remained an option for electric rates.

It’s obvious that both regulators and energy executives are frozen in time and they know it. They spend much of their time blaming each other for the delays. The blame game continues unabated at many industry events. The pace, ambiguity, and inconclusiveness of this regulatory drama seem to be a reenactment of the play Waiting for Godot.

The Missing Customer

For all practical purposes, utilities think of the regulator as their main customer. The end‐​use customer is almost an afterthought, consigned to being a “ratepayer” or “meter.” Whatever innovations take place on customers’ premises are referred to as “behind the meter.” Imagine how Nordstrom’s would thrive if it refused to consider what happens “behind the cash register.”

The regulators, in turn, often think of the legislature or the governor as being their main customer. The elected officials have their eyes on the next election. Their final customer, the American voter, is actually the utility’s customer and that’s how the circle is completed.

As we all know, emotion trumps logic when it comes to winning votes and often leads to unsustainable energy policies and unrealistic timetables. Elected officials change every few years and regulators often change every few years. Depending on the frequency of the crises that routinely afflict utilities during these tempestuous times, utility CEOs also often change every few years. That’s chaos theory in action.

It used to be said that rate design is more art than science. In fact, just last year, that notion was put to me in a regulatory hearing where we were discussing the case for demand charges. I said the notion was mostly rooted in politics. The whole room broke out in laughter.

Earlier, I had been grilled for 90 minutes by one of the commissioners. After the cross‐​examination ended, a person came up to me and said that I should write a book about these encounters. I said I have certainly had my share, trying to push regulators and utilities to listen to their customers.

A couple of years ago, I asked a newly appointed regulator in a large western state how independent of state government the commission’s policies would be. She said that she and her colleagues respected their chief executive very much. I said that was not my question. She asked me to be more specific. Because that state has more solar panels than any other state, I asked her when we should expect to see a change in net energy metering policies. Her answer left me stunned: “You know that the solar lobby in the state is very powerful.”

Time for Change

As a freshman at the University of Karachi in 1969, I came across Paul Samuelson’s Economics textbook. Every chapter began with a quote. One that has stayed with me is from Lewis Carroll:

The time has come, the Walrus said

To talk of many things:

Of shoes — and ships — and sealing wax

Of cabbages — and kings;

And why the sea is boiling hot;

And whether pigs have wings.

While every state is in a big rush to move ahead with decarbonization and has specified some very aggressive timelines for becoming 100% decarbonized, just about all the policy solutions are on the supply side. There is almost no inclusion of dynamic load flexibility, which could help deal with the intermittent nature of renewable energy.

For those of us who work in the electric utility industry, the time has come to rethink regulation, reimagine the utility, and reconnect with the real customer. That journey can no longer be delayed.

The best way I can think of beginning this journey is to make “customer‐​centricity” the guiding principle. This means leaving the past behind and focusing on the future. It does not mean simply creating a new website or sending frequent text messages to customers. Nor does it mean just engaging in social norming to shape customer behavior. It means changing the culture of the industry, reimagining utilities as service providers, hiring staff with an open mindset and new skills, reaching out to customers to understand their changing needs, and developing new products and services to meet those needs.

This journey will involve finding new ways to engage with customers and observing those customers in real time to understand their energy‐​buying decisions. Unless these steps are undertaken, the customer is going to leave both the utility and the regulator in the dust.