Flicking a circuit breaker to power cycle hundreds of desktop computers inside interactive museum exhibits is hardly ideal. Computers tend to get cranky when improperly shutdown, and there’s an non-zero risk of data loss. However, financial concerns ruled out commercial computer management solutions, and manually shutting down each exhibit at the end of the day is not practical. Tasked with finding a solution, [Jeff Glass] mixed off-the-shelf UPS (uninterruptible power supply) hardware, a Featherwing and some Python to give the museum’s computer-run exhibits a fighting chance.
Without drastically changing the one-touch end-of-day procedure, the only way to properly shutdown the hundreds of computers embedded in the museum exhibits involved using several UPS units, keeping the PCs briefly powered on after the mains power was cut. This in itself solves nothing – while the UPS can trigger a safe shutdown via USB, this signal could only be received by a single PC. These are off-the-shelf consumer grade units, and were never intended to safely shut down more than one computer at a time. However, each 300 watt UPS unit is very capable of powering multiple computers, the only limitation is the shutdown signal and the single USB connection.
To get around this, the Windows task scheduling service was setup to be triggered by the UPS shutdown signal, which itself then triggered a custom Python script. This script then relays the shutdown signal from the UPS to every other computer in the museum, before shutting itself down for the evening.
While many computers can be enabled to boot on power loss, the UPS and safe shutdown scripts meant that this wasn’t an option. To get around this, an ESP32 Featherwing and a little bit if CircuitPython code sends out WOL (wake-on-LAN) signals over Ethernet automatically on power up. This unit is powered by a non-UPS backed power outlet, meaning that it only sends the WOL signal in the morning when mains power is restored via the circuit breaker.
There are undoubtedly a variety of alternative solutions that appear ‘better’ on paper, but these may gloss over the potential costs and disruption to a multi-acre museum. Working within the constraints of reality means that the less obvious fix often ends up being the right one. How would you have tackled this problem? Sound off in the comments below. And while you’re here, make sure to check out our coverage of other UPS solutions, like this supercap UPS.
Feels a lot more complicated than simply issuing remote management commands via active directory for the windows machines or just ssh on demand without needing the power off signal.
The WoL trick is clever!
Yeah, maybe they don’t have this, but I thought most BIOS anymore had scheduling where you could power off/bring online at specific times. Not centrally managed, but maybe not something you have to touch very often.
Nah, there are several ways to get Windows to shutdown. You can issue a remote command without having anything fancy like a domain controller.
This problem definitely could have been solved for $0.
To a museum a one time donation of under $50 of hardware is essentially $0
The UPS is desirable as a UPS anyway. And its always easier to not change what the staff does: see example from the show the IT Crowd
Moss: I turn *on* the soldering Iron and walk away. -> fire is started.
UPS batteries don’t last forever if they are used for daily shutdowns. The batteries often are asked to discharge at a few times their C rating in a UPS.
Scheduled task: shutdown -s -t 60
Unless I’m missing something the shutdown procedure is way over complicated.
I would have just setup Windows task scheduling service on each computer to shutdown the computers at the same closing time everyday. If the circuit breaker is the last thing you do. They will all be safely shutdown at the same time everyday before you drop the breaker.
As said above tho, the power-on via WOL is clever…
Then you have to reprogram all the computers when the museum changes its schedule. Perhaps there staying open late on Friday once a month for a private fundraiser. They have an overnight kids program once a year, etc.
While this is probably not an optimum solution, it is flexible.
Looks great!
If the computers are already on a network, they could also try Ansible to shutdown remotly every single computer.
It’s agentless, so it only needs an ssh user configured in the clients
If they used even only one USP with LAN card (event the cheaper ones can have LAN card addon), they could hook the power-loss-shutdown command to all PCs at once.
Give away passes to local students and get all the free labor you need. Students will do just about anything to get cred for college.
Switching things on/off isn’t the best idea. Makes material brittle due to mechanical stress that’s caused
by expansion through heat/shrinking through lack of heat.
Also applies to electronics and ICs, even CRT monitors (use screensavers instead) .
For electronics, it’s best to run at a constant temperature that’s comfy for 24/7.
Breakers also aren’t designed to be used as a light switch. They aren’t designed for that kind of duty cycle. Unless you’re being dumb, a breaker is probably only going to trip a dozen or less times in it’s useful lifetime
Is ‘nut’ not available for Windows? Sad.
I’ve been dreaming about a Windows WMI Battery device (https://docs.microsoft.com/en-us/windows/win32/cimwin32prov/win32-battery) that is underpinned by a NUT network server. Seems like you could then turn your devices into a “laptop”, and then each can simply react to the “on battery event” and shutdown immediately.
The NUT server then doesn’t even have to be a real one (for this case), it could be a simulation that emits the “on battery event” when a button is pressed. (This could even be built into an ESP32 if that is your desire, since the NUT protocol is so simple!) In that way, you don’t even have to stress the battery of the UPS by powering the computers even for a couple of minutes. You could even have the simulation be a proxy to the real UPS, such that a genuine “on battery” event would be passed through to the computers, triggering the shutdown anyway.
Just wish I knew more about Windows development, so that I could make sense of the WMI examples!
I’d probably make their drives read-only and skip the safe shutdown. That also protects them against any number of other issues that might occur
Smells very heavily of no/poor research.
As mentioned earlier, there’s NUT it Networked Upsd, but even powerchute supported this decades ago.
Even a daily scheduled task to shutdown would be plebtyful.
If however no additional software is to be added to the running systems, I suppose a USB shutdown solution would do. I wonder though if it works without drivers. HID power button event probably …
But true, hackers gotta hack ;)
>HID power button event probably
Don’t think power switch is connected via HID. It got to be hooked up somewhere in either chipset or the SuperIO as it act shutdown a PC even when it is locked up/blue screen or in BIOS.
Or USB dongle that acts like Rubber Ducky/BadUSB. i.e. it’ll do the following:
– Press the windows logo + r, to open up the Run dialogue box
– Type cmd and press enter
– Type your command and press enter
The command being the dos shell shutdown /s command.
Task schedule a shutdown /s /f to get the computers to shut down. Configure the PCs to Boot On Power (sometimes called Boot After Power Loss). The actual power lines have to be shut off after the PCs are down and turned on in the morning by the museums power system though. Easy. How do I know? We have been using this system for years in 50+ museums that I have worked for.
precisely this. I had to scroll too far to find this comment.
Heck, I’d just replace the desktop computers with laptops, that have their own built-in UPS, and wouldn’t take up as much space.
Yeah because they really want to buy loads of laptops and in a few years time have to replace batteries
Also any kind of repair is harder
Exactly. Or raspberry pi
Seems like one can also power off PC using remote commands / scheduled shutdown. While setting the BIOS to always power on PC when power restores. This should solve the problem without needing UPS? But you will need to connect all PCs to be shutdown to the same set of breakers.
If you insist on a physical solution:
Most UPS (also/especially: ancient) software works with serial cable signaling (usually RTS/CTS line), so you only need one cable with multiple connectors for the signal.
Or one could use a simple USB keyboard controller and one shared for the “power key pressed” signal.
Well. Consider this: It´s not a computer classroom here. It´s a museum: large rooms with displays sprinkled here and there and the computer hidden in some box. So I´m afraid running your RTS/CTS line around the building is not only very inconvenient, but would not work at all.
did you read the article? they have multiple machines connected to single UPS. How exactly is connecting all the computers via serial any more inconvenient than the way they are connected via power cables?
Pretty sure apcupsd has a windows client. You just need the one on the ups usb or serial cable to run as master and the others connected to the same ups configured as slave. I use a setup like that in my server rack to shut down multiple servers and only use 1 usb cable.
And really, ups units should be a safety feature, not a primary power switch. Either use something like ssh to remotely power them down or use task scheduler to turn off the computers on a schedule, preferably well at least 10 minutes before you plan to hit the breaker.
I assume they had some reason not to use
Shutdown /m \\labpc1 /s /t 0
Or it’s cousin…
For /t %i in (machines.txt) do shutdown /m \\%i /s /t 0
How about having everything virtualized? Have a rack or two with plenty of cpu and gpu power, and run io thru fiber.
Should be no problem running scripts to restart VM every night.
For existing hardware, have them on domain and have scheduled tasks deployed to shutdown when needed to and update time from domain controller when necessary. WOL in the morning.
Hi Y’all,
A lot of good comments and suggestions here! I’ve added a follow-up at the end of the original post responding to the most common suggestions, some of which were reacting to context I left out of the original project description. Hope it’s useful for anyone who finds this thread in posterity.
Jeff Glass (Author)
I worked at a museum for five years building exhibits with computers in them. We also sold exhibits and built traveling exhibits. “Flicking a master switch” to shut everything down was the norm everywhere, and you’d be surprised how often this was totally fine. Things rarely became corrupt. Computers had to be able to power on with a single power strip and shot down by pulling the plug and still work. They did not have network connections, were not managed, and were easily replaceable. That’s the industry as I knew it… and it worked.