Back in October 2018, a bombshell rocked the tech industry when Bloomberg reported that some motherboards made by Supermicro had malicious components on them that were used to spy or interfere with the operation of the board, and that these motherboards were found on servers used by Amazon and Apple. We covered the event, looking at how it could work if it were true. Now seven months have passed, and it’s time to look at how things shook out.
If you’ve never seen an IBM AS/400 machine, don’t feel bad. Most people haven’t. Introduced in 1988 as a mid-range server line, it used a unique object-based operating system and was geared specifically towards business and enterprise customers. Unless you’re a particularly big fan of COBOL you probably won’t have much use for one today, but that doesn’t mean they aren’t worth playing around with if the opportunity presents itself.
So when a local IT company went belly up and was selling their old hardware, including a late 90’s era IBM AS/400e Series, [Rik te Winkel] jumped at the chance to take this unique piece of computing history home. He knew it was something of a risk, as maintenance and repair tasks for these machines were intended to be done by IBM certified technicians rather than the DIYer, leaving little in the way of documentation or even replacement parts. But in the end it worked out, and best of all, he documented the successful process of dragging this 90’s behemoth into the blinding light of the twenty-first century for all the world to see.
After getting the machine home and sitting through its thirty minute boot process, [Rik] was relieved to see the code
01 B N pop on the server’s display. This meant the system passed all the internal checks and was ready to go, he just had to figure out how to talk to the thing. Built to be a pure server, the machine didn’t offer any video output so he’d have to log into it over the network.
[Rik] noted that there was no new DHCP entry in his router for the server, but of course that was hardly surprising as the machine would have certainly had a static IP when it was in use. So he shut the server down, plugged it directly into his laptop’s Ethernet port, and watched the output of Wireshark as it went through its arduous boot sequence. Eventually he started to pick up packets coming from the IP address 10.10.10.9, and he had his target.
There are a few clients out there that allow you to remotely log into an AS/400, so he downloaded one and pointed it to the server’s IP. He was surprised to see the operating system was apparently in Dutch, but at least he was in. He tried a few common usernames and passwords, helped along by the fact that this OS from a somewhat more innocent era will actually tell you if you have the username right or wrong, and eventually managed to hack the Gibson with the classic admin/admin combo.
So he was in, but now what? [Rik] decided that he couldn’t truly call this machine bested until he could pull up the Hackaday Retro Edition, so he started work on writing a program to let him pull down the page directly on the AS/400 in IBM’s proprietary Report Program Generator (RPG) programming language. You know, as one does. He didn’t quite feel up to writing a whole HTML parser, but he got as far as generating a HTTP GET request, downloading the page’s source, and opening it up as a local file. That’s good enough for us.
Our very own [Al Williams] documented his adventures poking around an Internet-connected AS/400 machine, which might serve as a helpful primer if you ever find one of these delightfully oddball computers kicking around the local recycling center.
The failed launch of Soyuz MS-10 on October 11th, 2018 was a notable event for a number of reasons: it was the first serious incident on a manned Soyuz rocket in 35 years, it was the first time that particular high-altitude abort had ever been attempted, and most importantly it ended with the rescue of both crew members. To say it was a historic event is something of an understatement. As a counterpoint to the Challenger disaster it will be looked back on for decades as proof that robust launch abort systems and rigorous training for all contingencies can save lives.
But even though the loss of MS-10 went as well as possibly could be expected, there’s still far reaching consequences for a missed flight to the International Space Station. The coming and going of visiting vehicles to the Station is a carefully orchestrated ballet, designed to fully utilize the up and down mass that each flight offers. Not only did the failure of MS-10 deprive the Station of two crew members and the experiments and supplies they were bringing with them, but also of a return trip which was to have brought various materials and hardware back to Earth.
But there’s been at least one positive side effect of the return cargo schedule being pushed back. The “Spaceborne Computer”, developed by Hewlett Packard Enterprise (HPE) and NASA to test high-performance computing hardware in space, is getting an unexpected extension to its time on the Station. Launched in 2017, the diminutive 32 core supercomputer was only meant to perform self-tests and be brought back down for a full examination. But now that its ticket back home has been delayed for the foreseeable future, NASA is opening up the machine for other researchers to utilize, proving there’s no such thing as a free ride on the International Space Station.
As far as computer architectures go, ARM doesn’t have anything to be ashamed of. Since nearly every mobile device on the planet is powered by some member of the reduced instruction set computer (RISC) family, there’s an excellent chance these words are currently making their way to your eyes courtesy of an ARM chip. A userbase of several billion is certainly nothing to sneeze at, and that’s before we even take into account the myriad of other devices which ARM processors find their way into: from kid’s toys to smart TVs.
ARM is also the de facto architecture for the single-board computers which have dominated the hacking and making scene for the last several years. Raspberry Pi, BeagleBone, ODROID, Tinker Board, etc. If it’s a small computer that runs Linux or Android, it will almost certainly be powered by some ARM variant; another market all but completely dominated.
It would be a fair to say that small devices, from set top boxes down to smartwatches, are today the domain of ARM processors. But if we’re talking about what one might consider “traditional” computers, such as desktops, laptops, or servers, ARM is essentially a non-starter. There are a handful of ARM Chromebooks on the market, but effectively everything else is running on x86 processors built by Intel or AMD. You can’t walk into a store and purchase an ARM desktop, and beyond the hackers who are using Raspberry Pis to host their personal sites, ARM servers are an exceptional rarity.
Or at least, they were until very recently. At the re:Invent 2018 conference, Amazon announced the immediate availability of their own internally developed ARM servers for their Amazon Web Services (AWS) customers. For many developers this will be the first time they’ve written code for a non-x86 processor, and while some growing pains are to be expected, the lower cost of the ARM instances compared to the standard x86 options seems likely to drive adoption. Will this be the push ARM needs to finally break into the server and potentially even desktop markets? Let’s take a look at what ARM is up against.
Running a server completely off solar power seems like it would be a relatively easy thing to do: throw up a couple of panels, tack on a charge controller and a beefy battery, and away you go. But the reality is somewhat different. Most of us hackers are operating on a relatively limited budget and probably don’t have access to the kind of property you need to put out big panels; both pretty crippling limitations. Doing solar on a small-scale is hard, and unless you really plan ahead your setup will probably be knocked out on its first cloudy day.
So when [Kris de Decker] wanted to create a solar-powered version of his site “Low-tech Magazine”, he went all in. Every element of the site and the hardware it runs on was investigated for potential power savings, and luckily for us, the entire process was written up in meticulous detail (non-solar version here). The server still does go down from time to time if the weather is particularly poor, but in general it maintains about 90% uptime in Barcelona, Spain.
The solar side of the equation is fairly simple. There’s a 50 watt photovoltaic panel charging a 12V 7Ah lead-acid battery though a 20A charge controller. With an average of 4 to 6 hours of sunlight a day, the panel generates 300 Wh of electricity in the best case scenario; which needs to be split between charging the battery and running the server itself.
As for the server, [Kris] chose the Olimex Olinuxino A20 Lime 2 in part because of it being open source hardware, but also because it’s very energy-efficient and includes a AXP209 power management chip. Depending on processor load, the Olimex board draws between 1 and 2.5 watts of power, which combined with charging losses and such means the system can run through two days of cloudy weather before giving up the ghost. A second battery might be added in the future to help improve the run time during low-light conditions, but for now its been working pretty well.
Perhaps the most interesting part of the whole project are the lengths to which the website itself was optimized to keep resource utilization as low as possible. Images are compressed using dithering to greatly reduce their file sizes, and the site eschews modern design in favor of a much less processor intensive static layout. There’s even a battery capacity display integrated into the page through some clever use of CSS. Even if you aren’t looking to set up your own sun worshiping website, there are tips here for building efficient web pages that could absolutely be put to use in other projects.
If you’re interested in solar projects, we’ve got you covered. From an open source charge controller to building DIY photovoltaic panels, there’s plenty of prior art you should find very…illuminating. Please clap.
This morning Bloomberg is reporting a bombshell for hardware security. Companies like Amazon and Apple have found a malicious chip on their server motherboards. These are not counterfeit chips. They are not part of the motherboard design. These were added by the factory at the time of manufacture. The chip was placed among other signal conditioning components and is incredibly hard to spot as the nature of these motherboards includes hundreds of minuscule components.
Though Amazon and Apple have denied it, according to Bloomberg, a private security contractor in Canada found the hidden chip on server motherboards. Elemental Technologies, acquired by Amazon in 2015 for its video and graphics processing hardware, subcontracted Supermicro (Super Micro Computer, Inc.) to manufacture their server motherboards in China. It is unknown how many of the company’s products have this type of malicious hardware in them, equipment from Elemental Technologies has been supplied to the likes of government contractors as well as major banks and even reportedly used in the CIA’s drone operations.
How the Hack Works
The attacks work with the small chip being implanted onto the motherboard disguised as signal couplers. It is unclear how the chip gains access to the peripherals such as memory (as reported by Bloomberg) but it is possible it has something to do with accessing the bus. The chip controls some data lines on the motherboard that likely provide an attack vector for the baseboard management controller (BMC).
Hackaday spoke with Joe FitzPatrick (a well known hardware security guru who was quoted in the Bloomberg article). He finds this reported attack as a very believable approach to compromising servers. His take on the BMC is that it’s usually an ARM processor running an ancient version of Linux that has control over the major parts of the server. Any known vulnerability in the BMC would be an attack surface for the custom chip.
Data centers house thousands of individual servers that see no physical interaction from humans once installed. The BMC lets administrators control the servers remotely to reboot malfunctioning equipment among other administrative tasks. If this malicious chip can take control of the BMC, then it can provide remote access to whomever installed the chip. Reported investigations have revealed the hack in action with brief check-in communications from these chips though it’s difficult to say if they had already served their purpose or were being saved for a future date.
Adding hardware to a design is fundamentally different than software-based hacking: it leaves physical evidence behind. Bloomberg reports on US government efforts to investigate the supply chain attached to these parts. It is worth noting though that the article doesn’t include any named sources while pointing the finger at China’s People’s Liberation Army.
The solution is not a simple one if servers with this malicious chip were already out in the field. Even if you know a motherboard has the additional component, finding it is not easy. Bloomberg also has unconfirmed reports that the next-generation of this attack places the malicious component between layers of the circuit board. If true, an x-ray would be required to spot the additional part.
A true solution for high-security applications will require specialized means of making sure that the resulting product is not altered in any way. This hack takes things to a whole new level and calls into question how we validate hardware that runs our networks.
Update: We changed the penultimate paragraph to include the word if: “…simple one if servers with…” as it has not been independently verified that servers were actually out in the field and companies have denied Bloomberg’s reporting that they were.
[Note: Image is a generic photo and not the actual hardware]
Once upon a time, [hardwarecoder] acquired a Gen8 HP microserver that he began to toy around with. It started with ‘trying out’ some visualization before spiraling off the rails and fully setting up FreeBSD with ZFS as a QEMU-KVM virtual machine. While wondering what to do next, he happened to be lamenting how he couldn’t also fit his laptop on his desk, so he built himself a slick, motion-sensing KVM switch to solve his space problem.
At its heart, this device injects DCC code via the I2C pins on his monitors’ VGA cables to swap inputs while a relay ‘replugs’ the keyboard and mouse from the server to the laptop — and vice-versa — at the same time. On the completely custom PCB are a pair of infrared diodes and a receiver that detects Jedi-like hand waves which activate the swap. It’s a little more complex than some methods, but arguably much cooler.
Using an adapter, the pcb plugs into his keyboard, and the monitor data connections and keyboard/mouse output to the laptop and server stream out from there. There is a slight potential issue with cables torquing on the PCB, but with it being so conveniently close, [hardwarecoder] doesn’t need to handle it much.