picture of water treatment facility with a hologram of hacker depicting cybersecurity incident

Job Site Network Security Lessons From the Minnesota Water Hack

August 11, 20266 min read

Nobody broke into Minnesota's water. They just logged in.

Four days before hackers locked operators out of more than 30 Minnesota water systems, the federal government told them exactly how it was going to happen.

CISA updated joint advisory AA26-097A on July 22, 2026. The first version went out April 7. Both said the same thing. Nation-state actors are scanning the public internet for the small industrial controllers that run pumps and valves, and when they find one, they change its password and its IP address so the operators can't get back in.

On July 26 and 27, that is what happened. Utilities that had read the advisory and pulled their controllers off the internet were fine. The ones that hadn't lost the ability to monitor and control their own plants.

The gap between those two groups is the entire story, and it has nothing to do with how skilled the attackers were.

What actually happened

The attackers reached the controllers over EtherNet/IP on port 44818, a port that was answering the open internet. Rockwell Automation and Allen-Bradley MicroLogix 1100 and 1400 units were hit, along with ControlLogix, plus Siemens S7-1200 and Schneider Electric Modicon M340 gear at other utilities. These are small computers that manage pumps, chemical dosing, and line pressure.

Once inside, they rewrote IP configurations and set administrative passwords, which locked out the people whose job it was to watch the plant. In some cases they went further and modified the control logic while leaving the operator screens showing normal readings, and disabled safety shutdown and alarm functions. Braham's treatment plant was disabled inside two hours. Plymouth had to run manually. Other sites saw pressure loss and flooding.

It did not stay in Minnesota. Reporting now puts the campaign across roughly a dozen states, including nine systems in Michigan plus utilities in Georgia, South Dakota, and New Jersey. Investigators have preliminarily attributed it to an Iranian-linked group, and they've been clear that the assessment could change.

Recovery is the part that should get a contractor's attention. Because the devices were locked and their addressing had been changed, nobody could log in the normal way to fix it. Field techs had to drive out, open the cabinet, put hands on the hardware, and reload a known-good copy of the configuration from an offline backup. Every utility that had one recovered. The rest rebuilt from memory.

They didn't pick the lock

We all picture the hacker hammering a keyboard, firing off some exotic exploit. This was the opposite. This was being waved through by the doorman.

Many of the controllers were sitting in modes that accept remote configuration by design. The attackers didn't need a trick. They asked, and the device handed them the same access it hands a plant engineer, because that is what it was built to do for anyone who could reach it.

Where an actual vulnerability was involved, it was an old one. CVE-2021-22681 is an authentication bypass in Rockwell Logix controllers, scored 9.8 out of 10, and it has no patch. The fix has always been to keep the device off the internet.

Here is the number that matters. As of July 30, after all of this was public, researchers still counted more than 4,100 Rockwell EtherNet/IP hosts, more than 4,100 Siemens S7-1200 hosts, and over 2,000 Schneider hosts answering from the open internet. The advisory went out in April. The attack was in July. The doors are still open.

You don't run a water plant. You run four of the same conditions.

I'm not going to tell you a job site trailer is a water treatment facility. It isn't. But strip this incident down and it rests on four conditions, and I find some mix of them on almost every site I walk.

A device reachable from the internet. Somebody port-forwarded the trailer router so the super could pull up cameras from his couch. The cellular modem came with a public IP and nobody changed anything. The camera recorder got put online so the GC could look in. Every one of those is a front door, and none of them were decisions anyone wrote down.

Remote configuration left on. Field gear ships in whatever mode makes install easiest. That is convenient on day one and it is a standing invitation on day two hundred. Nobody goes back and turns it off, because turning it off was never on anyone's list.

Firmware nobody patched. The router, the switch, the access point, the camera recorder. Ask who is responsible for updating those on your active sites. On most jobs the honest answer is no one, and the gear has been running whatever shipped on it since the trailer landed.

No offline copy of the configuration. This is the one that turns an incident into a week. If a device gets wiped or locked tomorrow, what are you restoring from? Minnesota's answer separated the utilities that were back up in a day from the ones that weren't. Nobody on a job site has that copy, because nobody was ever asked for it.

Somebody is going to ask you about this

Not a hacker. A carrier or a general contractor.

Cyber insurance renewals stopped being a formality. The questionnaire asks whether remote access requires multi-factor authentication, whether you patch on a schedule, and whether you can restore from backup. Answer those wrong and you find out at renewal what your premium really is, or that a claim gets argued instead of paid.

General contractors are doing the same thing further down. Security requirements land in the subcontract now, and the sub who can answer in one email keeps moving while the sub who has to go ask around does not.

If you have DoD work, there's a harder version of this. CMMC Level 1 has to come back MET on all 15 requirements, with no partial credit and no plan to fix it later, and boundary protection is the heaviest single piece of it. Every remote job site link counts as a boundary, including the LTE modem and the fixed wireless shot. That is a longer conversation and I've written about it separately.

For everybody else the point is simpler. The four conditions above are what somebody with leverage over your business is going to ask you about, and right now most contractors would have to guess at the answers.

Offline configuration backups, to be clear, are not on anybody's compliance checklist. I recommend them because they're the difference between a bad afternoon and a bad week.

Five things to check this week

  1. Find out what's exposed. Ask whoever set up each site which devices are reachable from outside the network. If nobody knows, that is your answer and your starting point.

  2. Kill the port forwards. Anything that was opened so someone could check on the site from home gets closed. Remote access runs through a gateway with multi-factor authentication or it doesn't run.

  3. Change every default and shared password: on routers, access points, switches, and camera recorders. Unique credentials per person, not one written inside the cabinet.

  4. Get the firmware current on all of it, and name the person who owns that going forward.

  5. Save a copy of every device configuration somewhere offline so a locked or wiped device is a drive out to the site and not a rebuild from scratch.

None of that requires new hardware and none of it requires a compliance budget. It requires somebody to own it.

If you'd rather have that be somebody other than you, that's the job I do. I'll walk your sites, tell you exactly what's answering the internet today, and show you what it takes to close it.

Don Petrocelly

Don Petrocelly

Don is the Founder and Principle Consultant of Don's Tech Rescue.

LinkedIn logo icon
Instagram logo icon
Youtube logo icon
Back to Blog

Ready to Get Your Tech Handled?

Start with a free Discovery Call and technology assessment. We'll look at your setup, office and job site, find the gaps, and give you a clear plan. No obligation, no pressure.

Proactive IT for AEC firms across Western Pennsylvania. We manage the tech your projects run on so you can run the job.

Services

  • Managed IT

  • Cybersecurity

  • Backup & Recovery

  • Network Management

  • Jobsite Connectivity

  • Micrsoft 365

Get in Touch

Copyright 2026. Don’s Tech Rescue. All Rights Reserved.