The Spanish Grid Drop-out - recently released information....

[Chris Jones]

On 12/05/2025 1:21 am, Bill Sloman wrote:

On 11/05/2025 8:38 pm, Chris Jones wrote:
On 11/05/2025 4:18 pm, Bill Sloman wrote:
I believe that there are some new regulations in at least one
Australian state, driven by the (fossil-fuel-stoked) fear of \"too
much solar destabilising the grid\", which require new home solar
inverters to stop exporting power, unless they receive continuous
\"permission to export\" signals from our overlords, the network
operators.

It is cruder than that. They\'ve just stopped paying any realistic
kind of feed-in tariff to people with roof-top solar, and as a result
40% of new roof-top solar in Australia is now being installed with
Tesla Powerwall or similar battery. It more than doubles the cost of
the installation, but reduces the pay-back time for the whole
installation to about seven years, and save you negotiating with your
power supplier about their derisory feed-in tariffs.


No, they say:

\"What happens if my solar inverter loses internet connectivity?

If your solar inverter loses internet connectivity, the excess energy
you export to the grid will automatically be reduced. This ensures it
can be safely managed.\"
( from here:
https://www.energy.vic.gov.au/__data/assets/pdf_file/0019/701911/Emergency-backstop-customer-factsheet-June-2024.pdf )

What they say isn\'t all that interesting. What they do is discourage
people from trying to sell their excess power back to the grid.

So if all of the inverters lose internet, which is entirely likely at
some point bearing in mind our telcos, we can expect a blackout too,
all so that \"it can be safely managed.\" The blackout will no doubt
help the telcos to get back online promptly. Fun times ahead.

I haven\'t lost my internet recently - the last time it happened it was
not due to anything the telcos had done - the mains supply to my
apartment block had to be cut off for hours while they replaced the
local distribution transformer, which sits just outside our front gate,
and it was entirely local. A few years back it dropped out for couple of
hours due a  problem with my telco, but it only affected people served
by that telco, and was confined to a single suburb.

The chance of all the inverters losing internet connectivity at once
doesn\'t seem to be all that high.

I think the chance of at least one major telco going offline in the next
decade is pretty high. It\'s happened to mobile networks and payment
systems several times.

For problems to occur, it isn\'t necessary that the inverters all lose
internet - the other end of the connection could also fail. If the other
end of the connection goes to just a few datacentres, and if for some
reason they get misconfigured, or just encounter a DNS problem, it might
cause an unnecessary blackout.

There are enough unavoidable causes of power failures, we needn\'t create
new ones through legislated fragility in otherwise resilient equipment.

They could at least build in a long random time delay between loss of
internet connection and the inverter shutting down (so the population of
inverters will gradually shut down over a 5+ hour period).

 

[Bill Sloman]

On 12/05/2025 10:20 pm, Chris Jones wrote:

On 12/05/2025 1:21 am, Bill Sloman wrote:
On 11/05/2025 8:38 pm, Chris Jones wrote:
On 11/05/2025 4:18 pm, Bill Sloman wrote:
I believe that there are some new regulations in at least one
Australian state, driven by the (fossil-fuel-stoked) fear of \"too
much solar destabilising the grid\", which require new home solar
inverters to stop exporting power, unless they receive continuous
\"permission to export\" signals from our overlords, the network
operators.

It is cruder than that. They\'ve just stopped paying any realistic
kind of feed-in tariff to people with roof-top solar, and as a
result 40% of new roof-top solar in Australia is now being installed
with Tesla Powerwall or similar battery. It more than doubles the
cost of the installation, but reduces the pay-back time for the
whole installation to about seven years, and save you negotiating
with your power supplier about their derisory feed-in tariffs.


No, they say:

\"What happens if my solar inverter loses internet connectivity?

If your solar inverter loses internet connectivity, the excess energy
you export to the grid will automatically be reduced. This ensures it
can be safely managed.\"
( from here:
https://www.energy.vic.gov.au/__data/assets/pdf_file/0019/701911/Emergency-backstop-customer-factsheet-June-2024.pdf )

What they say isn\'t all that interesting. What they do is discourage
people from trying to sell their excess power back to the grid.

So if all of the inverters lose internet, which is entirely likely at
some point bearing in mind our telcos, we can expect a blackout too,
all so that \"it can be safely managed.\" The blackout will no doubt
help the telcos to get back online promptly. Fun times ahead.

I haven\'t lost my internet recently - the last time it happened it was
not due to anything the telcos had done - the mains supply to my
apartment block had to be cut off for hours while they replaced the
local distribution transformer, which sits just outside our front
gate, and it was entirely local. A few years back it dropped out for
couple of hours due a  problem with my telco, but it only affected
people served by that telco, and was confined to a single suburb.

The chance of all the inverters losing internet connectivity at once
doesn\'t seem to be all that high.


I think the chance of at least one major telco going offline in the next
decade is pretty high. It\'s happened to mobile networks and payment
systems several times.

The whole point about ARPANET - and today\'s system is just a development
of that - is that it was resilient. People have managed to take down
bits of it anyway, but that\'s been just stupidity.

For problems to occur, it isn\'t necessary that the inverters all lose
internet - the other end of the connection could also fail. If the other
end of the connection goes to just a few datacentres, and if for some
reason they get misconfigured, or just encounter a DNS problem, it might
cause an unnecessary blackout.

But it is very unlikely to be quite so clumsily implemented.

There are enough unavoidable causes of power failures, we needn\'t create
new ones through legislated fragility in otherwise resilient equipment.

There doesn\'t seem to be any legislated fragility on offer - just people
trying to implem4ent a distributed system without paying enough
attention to what they are doing.

They could at least build in a long random time delay between loss of
internet connection and the inverter shutting down (so the population of
inverters will gradually shut down over a 5+ hour period).

Everybody and his brother has bright ideas about what might have
prevented the Spanish shut-down, uninhibited by any detailed low level
information about what actually went wrong.

Design is about developing a detailed understand of what is actually
going on - and in this case what actually went wrong, and only then
trying to change or improve the system to make it less likely to happen
again.

Ethernet relies on short random delays before you try to resend a
message. The wait times are chosen to match the delays around the
network. It might well be sensible to for the feed-in inverters on a
distributed grid to react to large phase inversions in a way that
doesn\'t lead to all of them dropping out at once.

It might be a whole lot more sensible for them to react by acting in a
way that tended to reduce the phase excursion. The amount they could do
would be constrained by the amount of power they could contribute, and
the amount of stored energy that they could call on to sustain any
corrective input.

Let\'s see some design ideas that might prevent the problem from
developing in the first place, rather than getting fixated on the kind
of mistake that might have created the problem.

--
Bill Sloman, Sydney

 

[Theo]

john larkin <jl@glen--canyon.com> wrote:

On Sat, 10 May 2025 11:22:20 -0700, Don Y
blockedofcourse@foo.invalid> wrote:

On 5/10/2025 9:58 AM, John Robertson wrote:
Perhaps for systems that have large solar or wind arrays they could use a
number of large rotating masses to smooth over these burps? Vacuum and magnetic
bearings...

I imagine a series of rotating masses so if any single or several fail
(earthquake, etc.) the system wouldn\'t collapse.

As you say, there is little inertia in these solar systems unlike water or fuel
generated power.

The sun is *still* shining. Why can\'t *it* supply the power
to all of the distributed inverters around the country at the
appropriate phase angle? You only need storage if your
actual source of power disappears, relative to the load.

I.e., turn excess generation capacity to \"braking mass\"

If every solar inverter was networked and controlled in
voltage/current/phase angle, by some intelligent system controller,
one might not be so dependant on rotating mass.

As solar and wind get to be dominant, micromanagement of power sources
and loads will be necessary to ensure uptime.

A naive question: why do we need to get these signals from the grid at all?
Why can\'t we broadcast a synchronisation message on something like LW radio
that is picked up by every generator large or small? Then the network
operator can monitor what\'s happening and adjust the signal as appropriate.

No need for internet connectivity means no problems with network delays,
only the speed of RF from one end of the country to the other. You would of
course have multiple transmitter sites - they would cost in terms of power
to run, but compared with grid power it\'s tiny. (before anyone says you
cannae get the transmitters any more, yes you can - Nautel will sell you
a new one)

Or is the problem that we actually do need slight desynchronisation - some
parts of the network become overloaded and need to \'slow down\' compared with
other parts? (and they do that by phase differences rather than voltage
sag) In which case the frequency differences follow the network topology
and the power flows.

Theo

 

[Martin Brown]

On 10/05/2025 17:58, John Robertson wrote:

On 2025-05-10 9:46 a.m., Bill Sloman wrote:
One of my LinkedIn contacts - an IEEE contact in this case - posted
some new data on LinkedIn, from a \"Simon Gallagher, Managing Director
at UK Networks Services | CEng | FIET | FEI | MBA \"

\"We have had an update from ENTSO-E on the Spanish complete power
failure. It is limited, but it helps to build the picture. I have
updated our charts with the new information.

Updated timeline:

1. Large generators in the South of Spain started to trip at 12:32:57
CET. Over a period of 20 seconds a total of 2.2GW was lost – this is
well beyond largest infeed so not secured against

2. The frequency looks to have been contained by system reserves until
what looks like a large trip at 12:33:16

3. At this stage, the frequency falls at about 0.5 Hz/s for 4 seconds,
until a rapid collapse starts
...

That is not unlike the failure in the UK Aug 2019 where an apparently
inconsequential minor power station dropping off due to a lightning
strike started a cascade failure that spread until they shed enough load
to get a balance again. Wide area power cut resulted.

Green energy systems often react badly to frequency deviations. Whilst
there is no reason why this should be the case it is frequently shown to
happen. Some BESS systems *are* configured to maintain grid frequency
but by no means all. They have the advantage of fast response but for
that to be true they must not also be already running at full capacity.

Problem in the UK is that daytime load is such that everything that can
is running close to the limits during the daytime with an evening peak
that stresses the N-S interconnectors even in summer. I noticed last
week one evening at peaktime that the supergrid power cables were
visibly sagging as a result of the current flowing through them. I\'d
never really noticed that before but I expect it happens all weekdays.

\"While I think a lack of inertia had an impact here, that does not
mean that the level of solar and wind was to blame - rather it is how
it has been integrated - more grid forming inverters, more rotating
mass is needed, I suspect.\"


Perhaps for systems that have large solar or wind arrays they could use
a number of large rotating masses to smooth over these burps? Vacuum and
magnetic bearings...

They are intrinsically dangerous if they store enough energy to really
matter. We had such a steel reinforced lead flywheel and motor generator
configuration on big radio telescopes storing just enough energy to stow
them in the event of a storm taking out 3 phase mains power. The dishes
can only reliably survive storms if they are pointed at the zenith.
(sometimes not even then)

Working out how far it would travel if it ever broke free from its very
substantial bearings was used as an exam question. It was installed
pointing so that it would not hit any property if it did.

I imagine a series of rotating masses so if any single or several fail
(earthquake, etc.) the system wouldn\'t collapse.

Magnetic levitation vacuum pumps were all the rage when I was in Japan.
That was until one day the entire world moved an inch to the left. Every
last one of them crashed with shattered titanium blades everywhere and
no vacuum/moist summer air in the chambers. Hell of a mess. After that
we went back to conventional bearings in all earthquake countries.

As you say, there is little inertia in these solar systems unlike water
or fuel generated power.

The advantage of gas turbines or diesel generators is that when the
rotor starts to slow it automatically increases the gas supply to try
and maintain frequency. The stored energy in the rotor is significant
but it it have the ability to output a bit extra or a bit less in
response to changing load that makes them so handy for stability.

The other alternative is to have loads of last resort that can be shed
at any time to compensate for loss of generating capacity, but losing
2.2GW in a single shot over 5s would severely test most networks.

--
Martin Brown

 

[Bill Sloman]

On 13/05/2025 12:36 am, Martin Brown wrote:

On 10/05/2025 17:58, John Robertson wrote:
On 2025-05-10 9:46 a.m., Bill Sloman wrote:
One of my LinkedIn contacts - an IEEE contact in this case - posted
some new data on LinkedIn, from a \"Simon Gallagher, Managing Director
at UK Networks Services | CEng | FIET | FEI | MBA \"

\"We have had an update from ENTSO-E on the Spanish complete power
failure. It is limited, but it helps to build the picture. I have
updated our charts with the new information.

Updated timeline:

1. Large generators in the South of Spain started to trip at 12:32:57
CET. Over a period of 20 seconds a total of 2.2GW was lost – this is
well beyond largest infeed so not secured against

2. The frequency looks to have been contained by system reserves
until what looks like a large trip at 12:33:16

3. At this stage, the frequency falls at about 0.5 Hz/s for 4
seconds, until a rapid collapse starts
...

That is not unlike the failure in the UK Aug 2019 where an apparently
inconsequential minor power station dropping off due to a lightning
strike started a cascade failure that spread until they shed enough load
to get a balance again. Wide area power cut resulted.

Green energy systems often react badly to frequency deviations. Whilst
there is no reason why this should be the case it is frequently shown to
happen. Some BESS systems *are* configured to maintain grid frequency
but by no means all. They have the advantage of fast response but for
that to be true they must not also be already running at full capacity.

Green energy systems draw power from wind and sun. Both are variable.
They have have to a maximum capacity way above their average load. They
won\'t ever all be running at full capacity.

Problem in the UK is that daytime load is such that everything that can
is running close to the limits during the daytime with an evening peak
that stresses the N-S interconnectors even in summer. I noticed last
week one evening at peaktime that the supergrid power cables were
visibly sagging as a result of the current flowing through them. I\'d
never really noticed that before but I expect it happens all weekdays.

\"While I think a lack of inertia had an impact here, that does not
mean that the level of solar and wind was to blame - rather it is how
it has been integrated - more grid forming inverters, more rotating
mass is needed, I suspect.\"


Perhaps for systems that have large solar or wind arrays they could
use a number of large rotating masses to smooth over these burps?
Vacuum and magnetic bearings...

They are intrinsically dangerous if they store enough energy to really
matter. We had such a steel reinforced lead flywheel and motor generator
configuration on big radio telescopes storing just enough energy to stow
them in the event of a storm taking out 3 phase mains power. The dishes
can only reliably survive storms if they are pointed at the zenith.
(sometimes not even then)

Working out how far it would travel if it ever broke free from its very
substantial bearings was used as an exam question. It was installed
pointing so that it would not hit any property if it did.

I imagine a series of rotating masses so if any single or several fail
(earthquake, etc.) the system wouldn\'t collapse.

Magnetic levitation vacuum pumps were all the rage when I was in Japan.
That was until one day the entire world moved an inch to the left. Every
last one of them crashed with shattered titanium blades everywhere and
no vacuum/moist summer air in the chambers. Hell of a mess. After that
we went back to conventional bearings in all earthquake countries.

As you say, there is little inertia in these solar systems unlike
water or fuel generated power.

The advantage of gas turbines or diesel generators is that when the
rotor starts to slow it automatically increases the gas supply to try
and maintain frequency. The stored energy in the rotor is significant
but it it have the ability to output a bit extra or a bit less in
response to changing load that makes them so handy for stability.

A battery energy storage system has exactly the same advantage. The
point about rotating lumps of metal is that they store energy. So does a
battery, and it has the advantage that it is less sensitive to earthquakes.

The other alternative is to have loads of last resort that can be shed
at any time to compensate for loss of generating capacity, but losing
2.2GW in a single shot over 5s would severely test most networks.

It probably wasn\'t a single 2.2GW source, but a badly configured
collection of smaller sources. The rest of the net does seem to have
been equally badly configured, but less tightly coupled.

We still haven\'t got a clue why the system fell over, but a common duff
algorithm is a minimal explanation.

--
Bill Sloman, Sydney

 

[john larkin]

On Sun, 11 May 2025 12:22:11 +1000, Chris Jones
<lugnut808@spam.yahoo.com> wrote:

On 11/05/2025 5:04 am, john larkin wrote:

As solar and wind get to be dominant, micromanagement of power sources
and loads will be necessary to ensure uptime.

This is largely unnecessary - if the control signal that was being sent
out by the central controller to micromanage each power source was
derived from a function of the frequency, phase, voltage etc., then
rather than trying to distribute the result of this calculation to
millions of devices with low latency, it is better to distribute just
the formula (once every few years or as necessary), and run it on a
microcontroller in the inverters several times every mains cycle. They
already have more than enough processing power.

A central (international!) controller would want to know what every
contributor was pushing into the grid, and probably see wind flow and
clouds moving around. One local transmission line could fail and take
down half of Europe. Again.

I believe that there are some new regulatuions in at least one Austrlian
state, driven by the (fossil-fuel-stoked) fear of \"too much solar
destabilising the grid\", which require new home solar inverters to stop
exporting power, unless they receive continuous \"permission to export\"
signals from our overlords, the network operators. In other words,
rather than exporting power in the case of communications failure, it
goes into the state of \"export no power\" in case of communications
failure, because otherwise people might unplug their internet to export
more scary solar power if exporting power was allowed when the internet
connection fails. This is a fairly new requirement, so not many
compliant devices are installed now, but once a few gigawatts of these
inverters are running, it will be interesting to see what happens when
there is a major internet outage on a hot summer day, and all of those
gigawatts suddenly go away. Hopefully they thought of that but I doubt it.

The rapid control algorithms should be distributed, and the only
low-latency communication signals they should rely upon are frequency
and voltage.

A solar panel with an algorithm can\'t now about potential system
overloads. Solar and wind will have to be shed sometimes to protect
the entire system. Loads shed too. Renewable-heavy grids are fragile.

 

[Joe Gwinn]

On Sat, 10 May 2025 11:58:51 -0700, john larkin <jl@glen--canyon.com>
wrote:

On Sun, 11 May 2025 02:46:34 +1000, Bill Sloman <bill.sloman@ieee.org
wrote:

One of my LinkedIn contacts - an IEEE contact in this case - posted some
new data on LinkedIn, from a \"Simon Gallagher, Managing Director at UK
Networks Services | CEng | FIET | FEI | MBA \"

\"We have had an update from ENTSO-E on the Spanish complete power
failure. It is limited, but it helps to build the picture. I have
updated our charts with the new information.

Updated timeline:

1. Large generators in the South of Spain started to trip at 12:32:57
CET. Over a period of 20 seconds a total of 2.2GW was lost – this is
well beyond largest infeed so not secured against

2. The frequency looks to have been contained by system reserves until
what looks like a large trip at 12:33:16

3. At this stage, the frequency falls at about 0.5 Hz/s for 4 seconds,
until a rapid collapse starts

4. By 12:33:21 the frequency has crashed to 48 Hz. At this stage the AC
interconnectors to France trip

5. Low Frequency Disconnect was activated, but looks to have had no
effect because 3 seconds later the system has collapsed completely

6. At 12:33:24 the system has completely collapsed, 27 seconds after the
first trip.

Some key comments from me:
- LFDD/UFLS seems to have had no impact on the fall of frequency, I
suspect RoCoF relays were operating by this stage, showing how unstable
the grid was

- I suspect a lack of rotating mass did mean that there was not enough
time for LFDD to have an impact

- A large divergence of frequency opened up between Spain and France for
about 5 seconds. This must have meant a very large phase angle and large
power flows

- The previous data that showed the frequency only dropping to 49 Hz
must have been a result of local generators kicking in where the
Gridradar devices were connected to the network (UPDATE this has now
been confirmed by Gridrader, their sensor in Malaga was switched over to
a UPS and then generator at 12:33:20.7, prior to the disconnection of
the Iberian Peninsula and therefore missing some of the frequency drop)\"

I haven\'t cut and pasted all of it. This paragraph struck me as interesting.

\"While I think a lack of inertia had an impact here, that does not mean
that the level of solar and wind was to blame - rather it is how it has
been integrated - more grid forming inverters, more rotating mass is
needed, I suspect.\"

Any hints at the precipating cause?

Maybe some modest local event triggered a fundamentally unstable
system.

This from the most recent Risks Digest:

Date: Sat, 10 May 2025 10:10:04 -0700
From: Rob Wilcox <robwilcoxjr@gmail.com>
Subject: Iberian Electric Grid Blackout 4/28/2025 Grid Engineering
Presentation (YouTube)

The electric grid is a complex system to deliver an invisible
commodity just in time between generators and each individual load,
safely and reliably. My interests include the grid operator interface,
control systems, markets, and the internal culture.

If the generation is greater than the load, the 50 or 60Hz frequency
increases, if the generation is less than the load the frequency
decreases. If the frequency is too high, or low, the generators
disconnect themselves to prevent mechanical damage. The control
systems also manage Voltage, and the relative phase of the Voltage and
current.

The real time operators watch over nested layers of distributed
control systems and have preplanned processes to bring the system back
to stability. If generators begin to take themselves offline, that
can lead to a cascading loss of more generators until the grid goes
dark, or divides into dark and operating islands.

Once the grid goes dark, the operators have preplanned processes to
open switches to make the dark areas into islands. Then the black
start generators are turned on as each island is energized, in exact
balance of load and generation.

Operators train on black start, generation control, failure response,
and planned maintenance switching on grid simulators. It has
similarities to airplane pilot training.

On Monday April 28, there was a large grid blackout. The grid is
instrumented with the state of every switch, loads, and generator
performance, and sub-second data from synchrophasors, so the data on
what happened is there to be analyzed. It is like a much, much, more
detailed flight data recorder.

Risks readers may enjoy this early readout on what is known so far in
very technical grid terminology. It is a good look into the culture of
the grid. The electricity system is a local monopoly. The result is
that there is continuous cooperation to improve it, rather than
competition found in other industries.

..<https://www.youtube.com/watch?v=LNStOXAsiDo>

Joe

 

[Don Y]

On 5/12/2025 7:31 AM, Theo wrote:

A naive question: why do we need to get these signals from the grid at all?
Why can\'t we broadcast a synchronisation message on something like LW radio
that is picked up by every generator large or small? Then the network
operator can monitor what\'s happening and adjust the signal as appropriate.

How have we managed to distribute electrical power over huge swaths of land
from multiple independent operators WITHOUT such a mechanism?

No need for internet connectivity means no problems with network delays,
only the speed of RF from one end of the country to the other. You would of

The actual path -- and the \"terrain\" over which it travels -- will determine
the propagation delay.

How does the system react to a (inevitable) loss of that control signal?

Inertia implies memory. Memory can be implemented in silicon -- with
whatever dynamic characteristics the modelers choose.

But, if you treat new cogeneration facilities as \"optional bolt-on products\",
you likely won\'t model the NEW system with them in place. Rather, you will
design them to disconnect from the OLD (existing) system if they \"feel\"
they can\'t cope with their current observations of that system\'s behavior.

course have multiple transmitter sites - they would cost in terms of power
to run, but compared with grid power it\'s tiny. (before anyone says you
cannae get the transmitters any more, yes you can - Nautel will sell you
a new one)

Or is the problem that we actually do need slight desynchronisation - some
parts of the network become overloaded and need to \'slow down\' compared with
other parts? (and they do that by phase differences rather than voltage
sag) In which case the frequency differences follow the network topology
and the power flows.

Theo

 

[piglet]

Don Y <blockedofcourse@foo.invalid> wrote:

On 5/12/2025 7:31 AM, Theo wrote:
A naive question: why do we need to get these signals from the grid at all?
Why can\'t we broadcast a synchronisation message on something like LW radio
that is picked up by every generator large or small? Then the network
operator can monitor what\'s happening and adjust the signal as appropriate.

How have we managed to distribute electrical power over huge swaths of land
from multiple independent operators WITHOUT such a mechanism?

No need for internet connectivity means no problems with network delays,
only the speed of RF from one end of the country to the other. You would of

The actual path -- and the \"terrain\" over which it travels -- will determine
the propagation delay.

How does the system react to a (inevitable) loss of that control signal?

Inertia implies memory. Memory can be implemented in silicon -- with
whatever dynamic characteristics the modelers choose.

But, if you treat new cogeneration facilities as \"optional bolt-on products\",
you likely won\'t model the NEW system with them in place. Rather, you will
design them to disconnect from the OLD (existing) system if they \"feel\"
they can\'t cope with their current observations of that system\'s behavior.

course have multiple transmitter sites - they would cost in terms of power
to run, but compared with grid power it\'s tiny. (before anyone says you
cannae get the transmitters any more, yes you can - Nautel will sell you
a new one)

Or is the problem that we actually do need slight desynchronisation - some
parts of the network become overloaded and need to \'slow down\' compared with
other parts? (and they do that by phase differences rather than voltage
sag) In which case the frequency differences follow the network topology
and the power flows.

Theo

Yes, in essence voltage and frequency are all that must to be monitored. I
think the unfortunate thing is to date most co-generation has been designed
to follow the grid but now their share is greater their algorithms need to
be updated to more closely participate in contributing to the grid rather
than passively following?

--
piglet

 

[Don Y]

On 5/12/2025 1:27 PM, piglet wrote:

Yes, in essence voltage and frequency are all that must to be monitored. I
think the unfortunate thing is to date most co-generation has been designed
to follow the grid but now their share is greater their algorithms need to
be updated to more closely participate in contributing to the grid rather
than passively following?

Exactly. As I said:

\"But, if you treat new cogeneration facilities as \"optional bolt-on
products\", you likely won\'t model the NEW system with them in place.
Rather, you will design them to disconnect from the OLD (existing)
system if they \"feel\" they can\'t cope with their current observations
of that system\'s behavior.\"

The networks characteristics /with cogeneration in place/ have to be
remodeled and the dynamics of how those cogenerators are expected
to behave has to be (iteratively) refactored into that model.

If you have lots of *tiny*, independent cogeneration facilities (e.g.,
rooftop solar), all the moreso as each of them can act without
involving others.

What if my system is TAKEN off-line (because I\'m having the roof repaired)?
That (those!) abrupt removal of capacity has to be anticipated instead
of some MW plant whose absence can be planned. Likewise for new capacity
brought on-line haphazardly (yet another rooftop system connected, today!)

You don\'t have to look at just \"now\" to make deductions about what
is happening in the network; you can remember what has happened
immediately prior (for some value of \"immediately\") and adjust
your response based on knowledge of what those observations tell you
about the network at large. Much the same way that this information
is inertially \"stored\" in a large mass.

 

[Jeroen Belleman]

On 5/12/25 21:42, Don Y wrote:

On 5/12/2025 7:31 AM, Theo wrote:
A naive question: why do we need to get these signals from the grid at
all?
Why can\'t we broadcast a synchronisation message on something like LW
radio
that is picked up by every generator large or small?  Then the network
operator can monitor what\'s happening and adjust the signal as
appropriate.

How have we managed to distribute electrical power over huge swaths of land
from multiple independent operators WITHOUT such a mechanism?

I don\'t think that using broadcast radio for real time mains
grid control is a good idea. It would be far too unreliable.

To first order, power plants adjust the power injected into
the grid by observing the grid frequency. When the frequency
drops, the injected power is increased.

This says nothing about the dynamic behaviour, which is far
more involved, and variable too. Apparently there are some
issues with that on the European grid.

Jeroen Belleman

[Snip...]

 

[Carlos E.R.]

On 2025-05-12 16:36, Martin Brown wrote:

On 10/05/2025 17:58, John Robertson wrote:
On 2025-05-10 9:46 a.m., Bill Sloman wrote:

....

Perhaps for systems that have large solar or wind arrays they could
use a number of large rotating masses to smooth over these burps?
Vacuum and magnetic bearings...

They are intrinsically dangerous if they store enough energy to really
matter. We had such a steel reinforced lead flywheel and motor generator
configuration on big radio telescopes storing just enough energy to stow
them in the event of a storm taking out 3 phase mains power. The dishes
can only reliably survive storms if they are pointed at the zenith.
(sometimes not even then)

Working out how far it would travel if it ever broke free from its very
substantial bearings was used as an exam question. It was installed
pointing so that it would not hit any property if it did.

It would affect earth rotation, too. Some huge water reservoir in China
is affecting it already.


....

--
Cheers, Carlos.

 

[Martin Brown]

On 12/05/2025 18:35, john larkin wrote:

On Sun, 11 May 2025 12:22:11 +1000, Chris Jones
lugnut808@spam.yahoo.com> wrote:

On 11/05/2025 5:04 am, john larkin wrote:

As solar and wind get to be dominant, micromanagement of power sources
and loads will be necessary to ensure uptime.

It only requires that enough of the larger more powerful systems
cooperate and that automatic load shedding occurs fast enough and with
the right amount to prevent cascade network failure when things go bad.
Spain seems to have got the latter catastrophically wrong.

UK wasn\'t too good in 2019 either.

This is largely unnecessary - if the control signal that was being sent
out by the central controller to micromanage each power source was
derived from a function of the frequency, phase, voltage etc., then
rather than trying to distribute the result of this calculation to
millions of devices with low latency, it is better to distribute just
the formula (once every few years or as necessary), and run it on a
microcontroller in the inverters several times every mains cycle. They
already have more than enough processing power.

They are all connected to the national grid. The grid frequency target
and voltage is extremely well known and all that is needed is for each
unit that can to try and drive the grid voltage and frequency towards
that target. Things get iffy when they drop out a lot of stuff all at
once because they are using the same rules and rapid collapse follows.

A central (international!) controller would want to know what every
contributor was pushing into the grid, and probably see wind flow and
clouds moving around. One local transmission line could fail and take
down half of Europe. Again.

The big fat controller is already needed for any national grid. The UK
once made the mistake of letting BBC TV into the main National Grid
control room live in the late 60\'s and the interviewer asked innocently
if the live displays meant that if everyone watching switched on their
kettle the needle would shift. An edict went out afterwards to the
effect of never again will any live broadcast team be allowed on site.

https://www.bbc.co.uk/programmes/b00vkjmy

Unfortunately the series where it is mentioned is no longer available to
stream.

The rapid control algorithms should be distributed, and the only
low-latency communication signals they should rely upon are frequency
and voltage.

The whole thing seems to be messy with the domestic ones made down to a
price being rather less able to cope with surprises. Only some of the
BESS systems are configured for frequency stabilisation. Their main
objective is to make money for their investors by time shifting power.

A solar panel with an algorithm can\'t now about potential system
overloads. Solar and wind will have to be shed sometimes to protect
the entire system. Loads shed too. Renewable-heavy grids are fragile.

It can sense if the voltage and/or frequency is too high or too low and
if it has output margin available act to counter it. This is only really
worthwhile if it has some stored battery energy reserve to draw upon.
The grid being overloaded is more common than over supplied (and there
are consumers of last resort that can load balance to some extent).

Wind power scaling as cube of windspeed means that quite often wind
Scottish wind farms are paid to feather their turbine blades because the
cables are far too feeble to carry the power away.

Big problem in daytime is that most of the BESS systems are running flat
out supplying power to industry and consumers so that they don\'t have a
lot of reserve to offer if things start to go wrong. Likewise when a
failure happens in the early evening peak consumption 6pm-8pm.

Peak demand premium pricing means they all want a slice of that cake.

--
Martin Brown

 

[john larkin]

On Mon, 12 May 2025 12:42:11 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

On 5/12/2025 7:31 AM, Theo wrote:
A naive question: why do we need to get these signals from the grid at all?
Why can\'t we broadcast a synchronisation message on something like LW radio
that is picked up by every generator large or small? Then the network
operator can monitor what\'s happening and adjust the signal as appropriate.

How have we managed to distribute electrical power over huge swaths of land
from multiple independent operators WITHOUT such a mechanism?

Big steam tubines with lots of spinning inertia and boilers full of
gigajoules of superheated water. And enough 24/7 local generation so
we don\'t need to import power from France.

And no commitments to take (or pay for) power that we don\'t need.

 

[Joe Gwinn]

On Mon, 12 May 2025 23:33:30 +0200, \"Carlos E.R.\"
<robin_listas@es.invalid> wrote:

On 2025-05-12 16:36, Martin Brown wrote:
On 10/05/2025 17:58, John Robertson wrote:
On 2025-05-10 9:46 a.m., Bill Sloman wrote:

...

Perhaps for systems that have large solar or wind arrays they could
use a number of large rotating masses to smooth over these burps?
Vacuum and magnetic bearings...

They are intrinsically dangerous if they store enough energy to really
matter. We had such a steel reinforced lead flywheel and motor generator
configuration on big radio telescopes storing just enough energy to stow
them in the event of a storm taking out 3 phase mains power. The dishes
can only reliably survive storms if they are pointed at the zenith.
(sometimes not even then)

Working out how far it would travel if it ever broke free from its very
substantial bearings was used as an exam question. It was installed
pointing so that it would not hit any property if it did.

It would affect earth rotation, too. Some huge water reservoir in China
is affecting it already.

Three Gorges Dam?

..<https://en.wikipedia.org/wiki/Three_Gorges_Dam>

Joe

 

[john larkin]

On Mon, 12 May 2025 14:24:27 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:

On 5/12/2025 1:27 PM, piglet wrote:

Yes, in essence voltage and frequency are all that must to be monitored. I
think the unfortunate thing is to date most co-generation has been designed
to follow the grid but now their share is greater their algorithms need to
be updated to more closely participate in contributing to the grid rather
than passively following?

Exactly. As I said:

\"But, if you treat new cogeneration facilities as \"optional bolt-on
products\", you likely won\'t model the NEW system with them in place.
Rather, you will design them to disconnect from the OLD (existing)
system if they \"feel\" they can\'t cope with their current observations
of that system\'s behavior.\"

The networks characteristics /with cogeneration in place/ have to be
remodeled and the dynamics of how those cogenerators are expected
to behave has to be (iteratively) refactored into that model.

If you have lots of *tiny*, independent cogeneration facilities (e.g.,
rooftop solar), all the moreso as each of them can act without
involving others.

Cogen means extracting power (like otherwise waste heat) from some
process.

https://en.wikipedia.org/wiki/Cogeneration

Rooftop solar is not often cogen.

What if my system is TAKEN off-line (because I\'m having the roof repaired)?
That (those!) abrupt removal of capacity has to be anticipated instead
of some MW plant whose absence can be planned. Likewise for new capacity
brought on-line haphazardly (yet another rooftop system connected, today!)

One rooftop solar array is parts-per-million of a national power
system. One more or less is unlikely to crash the system... although
it could.

One anchor dragged across the underwater feed from an offshore wind
farm could be interesting.

 

[Carlos E.R.]

On 2025-05-13 00:20, Joe Gwinn wrote:

On Mon, 12 May 2025 23:33:30 +0200, \"Carlos E.R.\"
robin_listas@es.invalid> wrote:

On 2025-05-12 16:36, Martin Brown wrote:
On 10/05/2025 17:58, John Robertson wrote:
On 2025-05-10 9:46 a.m., Bill Sloman wrote:

...

Perhaps for systems that have large solar or wind arrays they could
use a number of large rotating masses to smooth over these burps?
Vacuum and magnetic bearings...

They are intrinsically dangerous if they store enough energy to really
matter. We had such a steel reinforced lead flywheel and motor generator
configuration on big radio telescopes storing just enough energy to stow
them in the event of a storm taking out 3 phase mains power. The dishes
can only reliably survive storms if they are pointed at the zenith.
(sometimes not even then)

Working out how far it would travel if it ever broke free from its very
substantial bearings was used as an exam question. It was installed
pointing so that it would not hit any property if it did.

It would affect earth rotation, too. Some huge water reservoir in China
is affecting it already.

Three Gorges Dam?

.<https://en.wikipedia.org/wiki/Three_Gorges_Dam

Yep.

--
Cheers, Carlos.

 

[Bill Sloman]

On 13/05/2025 3:35 am, john larkin wrote:

On Sun, 11 May 2025 12:22:11 +1000, Chris Jones
lugnut808@spam.yahoo.com> wrote:

On 11/05/2025 5:04 am, john larkin wrote:

As solar and wind get to be dominant, micromanagement of power sources
and loads will be necessary to ensure uptime.

This is largely unnecessary - if the control signal that was being sent
out by the central controller to micromanage each power source was
derived from a function of the frequency, phase, voltage etc., then
rather than trying to distribute the result of this calculation to
millions of devices with low latency, it is better to distribute just
the formula (once every few years or as necessary), and run it on a
microcontroller in the inverters several times every mains cycle. They
already have more than enough processing power.

A central (international!) controller would want to know what every
contributor was pushing into the grid, and probably see wind flow and
clouds moving around. One local transmission line could fail and take
down half of Europe. Again.

What makes you think that?
The current control system clearly isn\'t that well informed, and it
works pretty much all the time.

I believe that there are some new regulatuions in at least one Australian
state, driven by the (fossil-fuel-stoked) fear of \"too much solar
destabilising the grid\", which require new home solar inverters to stop
exporting power, unless they receive continuous \"permission to export\"
signals from our overlords, the network operators. In other words,
rather than exporting power in the case of communications failure, it
goes into the state of \"export no power\" in case of communications
failure, because otherwise people might unplug their internet to export
more scary solar power if exporting power was allowed when the internet
connection fails.

This is nonsense. The Australian grid don\'t like having to deal with
excess power being exported by roof-top solar installation, and
discourage people from doing it, to the point where 40% of new roof-top
solar installations in Australia include a Tesla Powerwall or an
equivalent battery, and don\'t export anything.

This is a fairly new requirement, so not many
compliant devices are installed now, but once a few gigawatts of these
inverters are running, it will be interesting to see what happens when
there is a major internet outage on a hot summer day, and all of those
gigawatts suddenly go away. Hopefully they thought of that but I doubt it.

The rapid control algorithms should be distributed, and the only
low-latency communication signals they should rely upon are frequency
and voltage.

A solar panel with an algorithm can\'t know about potential system
overloads. Solar and wind will have to be shed sometimes to protect
the entire system. Loads shed too. Renewable-heavy grids are fragile.

The existing system doesn\'t know about potential system overloads, and
it works pretty much all the time. Regular grids are fragile too.

The renewable-heavy grid in South Australia was fragile, until they
bought the world first grid scale battery in 2017.

https://hornsdalepowerreserve.com.au/

They promptly devoted half the battery to short term phase correction
and frequency satabilisation, and it isn\'t fragile any more.

Carlos says that the Spanish mainland grid hasn\'t got any storage - no
grid battery and no pumped hydro, which is a bit silly.

--
Bill sloman, Sydney

 

[Don Y]

On 5/12/2025 2:33 PM, Carlos E.R. wrote:

Working out how far it would travel if it ever broke free from its very
substantial bearings was used as an exam question. It was installed pointing
so that it would not hit any property if it did.

It would affect earth rotation, too. Some huge water reservoir in China is
affecting it already.

\"OK, on the count of three, everybody JUMP...\"

 

[Bill Sloman]

On 13/05/2025 8:19 am, john larkin wrote:

On Mon, 12 May 2025 12:42:11 -0700, Don Y
blockedofcourse@foo.invalid> wrote:

On 5/12/2025 7:31 AM, Theo wrote:
A naive question: why do we need to get these signals from the grid at all?
Why can\'t we broadcast a synchronisation message on something like LW radio
that is picked up by every generator large or small? Then the network
operator can monitor what\'s happening and adjust the signal as appropriate.

How have we managed to distribute electrical power over huge swaths of land
from multiple independent operators WITHOUT such a mechanism?

Big steam turbines with lots of spinning inertia and boilers full of
gigajoules of superheated water. And enough 24/7 local generation so
we don\'t need to import power from France.

Spinning inertia and boilers fulln of super=heated steam are just energy
reservoirs. So are grid scale batteries. Getting excited about the
former and ignoring the latter is clear evidence of sloppy thinking.

> And no commitments to take (or pay for) power that we don\'t need.

Back in 2008 the book

https://en.wikipedia.org/wiki/Hot,_Flat,_and_Crowded

got published. It advanced the idea that electric car batteries could
help stabilise the grid (Tesla Powerwalls are just electric car batteries).

Nobody likes the idea that their assets are being used without their
explicit permission, but participants would be paid for their help.

--
Bill Sloman, Sydney