The Real AI Safety Discussion That Just Isn’t Happening

---

Support CleanTechnica’s work through a Substack subscription or on Stripe.

---

If you spend any time watching the AI safety debate play out online, you’ve probably noticed it’s a bit of a circus. Right now, the conversation is totally dominated by tech investors pushing for maximum speed, software developers who think a few lines of code can solve anything, and click-hungry influencers screaming about a digital apocalypse. 

There’s also the political side of this, especially when it comes to datacenters. As you’ll see in any politicized debate, some people are well-meaning while others are defending their own interests.Without performing a bunch of Vulcan mind melds, I can’t tell you for sure who’s on what side of that selfish/selfless divide, but I’m sure we all have our guesses that might be libelous if put in print, so I’ll steer clear of that in this article.

The latest thing we’re seeing is a push for a moratorium on datacenters until society gets a better grip on all aspects of AI. Not only are there safety issues (overblown by John Connor fantasies, of course), but there are questions of how society will run without abundant jobs, who gets the wealth from information that was taken out of the public domain, and how the whole industry should or shouldn’t be regulated.

But we’re missing a massive piece of the puzzle. 

There is an entire profession of risk and emergency management experts out there. These are the people who deal with complex, catastrophic failures in the real world every single day. They work in government agencies like FEMA, for most every county or parish in the US, and for many companies. They know how to prepare for, respond to, and lower the risks of disaster, and have been doing this for decades. 

Yet, when tech executives sit down to talk about keeping AI safe, those emergency managers and risk consultants are usually nowhere to be found.

We’re getting ready to integrate AI into critical physical infrastructure. We’re talking about regional power grids, virtual power plants, and autonomous EV networks. If we want to do this without causing a real-world disaster, the tech industry needs to stop trying to reinvent the wheel. We need to pull up a chair for the people who actually know how to handle a crisis and start borrowing heavily from established disaster management frameworks.

Moving Past “Perfect Code”

Right now, AI labs suffer from a massive tech delusion. They focus almost entirely on prevention. They want to align the model and build software guardrails so it never makes a catastrophic mistake.

While this sounds great on paper, emergency managers know that perfect prevention is a myth. Disasters happen anyway, because no human being has the infinite knowledge required to prevent all disasters. Complex systems inevitable fail, either due to an unforeseen flaw or because uncontrollable variables like weather and climate will push those systems over the edge and outside of what they were built to cope with. 

Instead of just trying to build an unhackable wall, emergency management generally relies on a four-phase disaster lifecycle:

• Mitigation: Reducing impacts before a failure happens by doing things like moving people and businesses out of flood danger.
• Preparedness: Getting ready for the inevitable failure by having supplies and trained people ready to step in and save lives and property.
• Response: What to do in the chaotic first 48 hours after the “bang”.
• Recovery: Getting systems and society back online safely and (ideally), moving back into the mitigation part of the cycle to “build back better” instead of just rebuilding the system that failed before.

Think of it like flood control. We spend billions on levee construction to prevent flooding. But we also have evacuation plans, swift-water rescue teams, and FEMA budgets because we know levees can fail. The AI industry is currently building the levee and completely ignoring the evacuation plan, the training, and the working relationships we learned were so valuable after 9/11.

Coordination When Things Go South

When a major tech failure happens right now, companies usually rely on chaotic PR scrambles and internal Slack chats. If a social media site goes down, that’s fine. But if an AI model managing a grid goes rogue, a Slack channel isn’t going to cut it because there are other stakeholders and decisionmakers who need to be in the loop.

After the 9/11 attacks, the US learned a very hard lesson about coordination. Different agencies and groups literally could not talk to each other because their radios and command structures didn’t match up. That disaster led to the widespread adoption of the Incident Command System (ICS) and the National Incident Management System (NIMS). 

Seeing how well the Pentagon response went compared to the initial chaos of the WTC site made it clear that people need to be working and practicing emergency plans regularly so that everyone knows they can work together naturally at the worst time.

We desperately need an ICS for AI, or at least to get AI companies into the loop on that. If an AI system managing a regional grid suffers a catastrophic failure, we need pre-built, working relationships between tech labs, utility operators, and government responders. Everyone needs to know exactly who is calling the shots the second the lights go out.

But, this is just one example. There are many different kinds of technological disasters that can occur. We don’t all have to be converted into paperclips or pressed into robo-slavery to have a very bad day. Coordination with the officials and experts that are already preparing for disasters is essential to finding those other risks, assessing them properly, and getting them into the cycle of mitigation, preparedness, response, and recovery.

High Risk, Low Frequency

Tech companies are great at tracking daily bugs. They push updates, watch the telemetry, and patch things on the fly. But they are terrible at modeling massive, rare physical failures.

In emergency management, there is a concept called HR/LF/NDT. That stands for High Risk, Low Frequency, Non-Discretionary Time.

Look at the 2021 Texas winter freeze. A deep freeze knocking out that much gas and wind generation at once was low frequency, but the risk was massive. When the grid started to physically collapse, operators found themselves in non-discretionary time. They had minutes to shed load or risk a total blackout that could have destroyed equipment and left the state dark for months.

When an AI model managing a virtual power plant experiences a cascading error, there’s no time to form a committee or wait for the CEO to draft a press release. You need split-second, non-discretionary decision-making protocols already drilled and in place.

You don’t get there by obsessing over perfect code or with moratoriums on new datacenters. You get there by identifying these dangerous tasks where there’s no thinking time and preparing for them until the response is almost like a reflex.

Resilience Over Prevention

Silicon Valley culture demands unhackable, perfectly aligned systems. But anyone who works with heavy infrastructure knows that perfect prevention is impossible in complex environments.

We need to build for resilience, not just prevention.

Remember the 2003 Northeast Blackout? That whole mess started with a software bug in a localized alarm system and a few overgrown trees. Because the grid lacked resilience, that tiny localized failure cascaded and knocked out power for 50 million people. Resilience means designing grids and EV networks so that a localized software failure physically cannot trip the whole coast. If an AI system goes rogue, the physical infrastructure needs the ability to isolate the problem, fail safely, and bounce back.

Think of it this way: some cities aim only to prevent floods with walls and dams. But, smart cities look for ways to redirect floods away from people, temporarily shut things off to prevent damage, and even add vegetation to soak up the moisture instead of sending it along on top of concrete to flood someone else. AI isn’t water, obviously, but keeping the risks of AI away from our vulnerabilities is an essential tactic that just isn’t discussed.

The Domino Effect

You can’t evaluate AI safety in a vacuum. We have to look at infrastructure interdependence.

A great example is the Colonial Pipeline ransomware attack. The hackers didn’t actually attack the physical pipeline or the pumps. They attacked the billing system. But because the systems were deeply interdependent, fuel stopped flowing to the East Coast. That caused panic buying, gas shortages, and transportation gridlock.

An AI failure in a virtual power plant doesn’t just knock out power. It knocks out EV charging networks. It takes down water pumps, cell towers, and emergency communications. A software glitch instantly becomes a multi-sector physical crisis.

I didn’t finish my degree in emergency management, but I spent many hours reading lengthy papers on this exact phenomenon. There are people who make a whole career out of mapping interdependencies, considering the philosophy of interdependent systems, and looking for better ways for people on the ground to untangle them or protect them from failing so drastically.

The papers they write might be dry and boring to the “move fast and break things” crowd, but they should at least be hiring people to look at their systems and the systems they’re connected to.

The Human Cost

There’s one last thing the tech world completely overlooks: the human factor.

Emergency responders train heavily for the psychological toll of a disaster. Air traffic controllers have specific protocols and support systems for dealing with the intense stress of near-misses or crashes.

AI lab workers currently have nothing comparable. If an AI system failure directly causes a massive blackout or physical harm, the engineers and lab workers on the other end of the screen are going to experience intense trauma. It’s predictable human nature, and learning techniques to get out of that “emotional basement” and back into clear thinking are essential.

Right now, the only thing tech companies prepare for them is a legal defense team and PR spin doctors. They need real emotional resilience training and support structures. At the very minimum, they need to learn about breathing techniques.

Wrapping It Up

AI safety isn’t just a software problem. The second AI touches the grid, the roads, or the water supply, it becomes a physical infrastructure and disaster management problem. Tech executives need to drop the ego, look outside of Silicon Valley, and hire seasoned emergency managers. At the very minimum, they should be making some phone calls and setting up a meeting with their county’s manager to discuss this and get the lines of communication open.

It’s much better to have them at the table now than to wait until the first major disaster forces the issue and unnecessary loss of life happens.