According to a new paper, a new AI model that optimizes the use of renewables and other energy sources outperforms traditional power restoration techniques for isolated microgrids.
It’s a story that’s all too common: high winds knock out a power line, leaving a community without power for hours or days at a time, which is inconvenient at best and dangerous at worst. Yu Zhang, Assistant Professor of Electrical and Computer Engineering at UC Santa Cruz, and his lab are using tools to improve the efficiency, reliability, and resilience of power systems, and they have developed an artificial intelligence (AI)-based approach for the smart control of microgrids for power restoration when outages occur.
In a new paper published in IEEE Transactions on Control of Network Systems, a top journal in the field of control systems and network science, they describe their new AI model and demonstrate that it outperforms traditional power restoration techniques. The paper’s first author is Shourya Bose, a Ph.D. student in Zhang’s lab.
“Nowadays, microgrids are really the thing that both people in industry and in academia are focusing on for the future power distribution systems,” said Zhang.
Essentially, we want to bring the power generation closer to the demand side in order to get rid of the long transmission lines. This can improve the power quality and reduce the power losses over the lines. In this way, we will make the grid smaller, but stronger and more resilient.
Yu Zhang
Many communities’ infrastructure and users rely entirely on a local power-generating utility company for electricity. This means that if a disaster or extreme weather event occurs, or even if a tree falls on a power line, power is cut off until repairs can be made.
Many electricity systems today are smart in the sense that they are linked to computers and sensors. Local renewable energy sources, such as rooftop solar panels or small wind turbines, are frequently used, and some households and buildings rely on backup generators and/or energy batteries to meet their electricity needs.
This mix of power sources presents an opportunity to address outages locally by using alternative energy sources to provide electricity before upstream power is restored. One way to do this is with a microgrid, which distributes electricity to small areas such as a few buildings or a town — although the size of the microgrid can vary.
The microgrid can be connected to the main power utility source, but also can function while disconnected in “islanding mode,” self-supported by alternate energy sources and unaffected by the issues impacting the main utility. Zhang’s research team focuses on optimizing how microgrids pull from these various alternate sources such as renewables, generators, and batteries to restore power quickly and correctly.
“Essentially, we want to bring the power generation closer to the demand side in order to get rid of the long transmission lines,” Zhang said. “This can improve the power quality and reduce the power losses over the lines. In this way, we will make the grid smaller, but stronger and more resilient.”
Zhang’s lab created an efficient framework that includes models of many components of the power system to optimally operate microgrids using an AI-based technique called deep reinforcement learning, which is the same concept that underpins large language models. Reinforcement learning is based on rewarding the algorithm for successfully responding to a changing environment, so an agent is rewarded when it successfully restores the required power of all network components. They explicitly model the real-world system’s practical constraints, such as the branch flows that power lines can handle.
“We’re modeling a whole bunch of things — solar, wind, small generators, batteries, and we’re also modeling when people’s electricity demand changes,” Bose said. “The novelty is that this specific flavor of reinforcement learning, which we call constrained policy optimization (CPO), is being used for the first time.”
Their CPO approach takes into account real-time conditions and uses machine learning to find long-term patterns that affect the output of renewables, such as the varying demand on the grid at a given time and intermittent weather factors that affect renewable sources. This is unlike traditional systems which often use a technique called model predictive control (MPC) that bases decisions simply on the available conditions at the time of optimization.
For example, if the CPO method predicts that the sun will shine brightly in an hour, it will use up its supply of solar energy knowing that it will be replenished later – a different strategy than it would take if the day was cloudy. It can also learn about the system based on long-term patterns of how the grid uses solar.
The researchers discovered that their CPO technique outperforms traditional MPC methods when forecasts of renewable sources are lower than reality due to a better understanding of all possible solar profiles throughout any given day.
They also discovered that in the event of a power outage, the reinforcement learning controller responds much faster than traditional optimization methods.
The research team recently demonstrated the effectiveness of their method by winning first place in a global competition in which participants were asked to use reinforcement learning or similar techniques to operate a power grid. The L2RPN Delft 2023 competition was co-sponsored by France’s electricity transmission system operator (Réseau de Transport d’Électricité), which the UC Santa Cruz researchers see as an indication that large-scale grid operators may now begin to shift toward AI and renewable energy techniques.
