This excellent content from the Hackaday writing crew highlights recurring topics and popular series like Linux-Fu, 3D-Printering, Hackaday Links, This Week in Security, Inputs of Interest, Profiles in Science, Retrotechtacular, Ask Hackaday, Teardowns, Reviews, and many more.
Most security professionals will tell you that it’s a lot easier to attack code systems than it is to defend them, and that this is especially true for large systems. The white hat’s job is to secure each and every point of contact, while the black hat’s goal is to find just one that’s insecure.
Whether black hat or white hat, it also helps a lot to know how the system works and exactly what it’s doing. When you’ve got the source code, either because it’s open-source, or because you’re working inside the company that makes the software, you’ve got a huge advantage both in finding bugs and in fixing them. In the case of closed-source software, the white hats arguably have the offsetting advantage that they at least can see the source code, and peek inside the black box, while the attackers cannot.
Still, if you look at the number of security issues raised weekly, it’s clear that even in the case of closed-source software, where the defenders should have the largest advantage, that offense is a lot easier than defense.
So now put yourself in the shoes of the poor folks who are going to try to secure large language models like ChatGPT, the new Bing, or Google’s soon-to-be-released Bard. They don’t understand their machines. Of course they know how the work inside, in the sense of cross multiplying tensors and updating weights based on training sets and so on. But because the billions of internal parameters interact in incomprehensible ways, almost all researchers refer to large language models’ inner workings as a black box.
And they haven’t even begun to consider security yet. They’re still worried about how to construct obscure background prompts that prevent their machines from spewing hate speech or pornographic novels. But as soon as the machines start doing something more interesting than just providing you plain text, the black hats will take notice, and someone will have to figure out defense.
Indeed, this week, we saw the first real shot across the bow: a hack to make Bing direct users to arbitrary (bad) webpages. The Bing hack requires the user to already be on a compromised website, so it’s maybe not very threatening, but it points out a possible real security difference between Bing and ChatGPT: Bing gives you links to follow, and that makes it a juicy target.
We’re right on the edge of a new security landscape, because even the white hats are facing a black box in the AI. So far, what ChatGPT and Codex and other large language models are doing is trivially secure – putting out plain text – but Bing is taking the first dangerous steps into doing something more useful, both for users and black hats. Given the ease with which people have undone OpenAI’s attempts to keep ChatGPT in its comfort zone, my guess is that the white hats will have their hands full, and the black-box nature of the model deprives them of their best hope. Buckle your seatbelts.
This article is part of the Hackaday.com newsletter, delivered every seven days for each of the last 200+ weeks. It also includes our favorite articles from the last seven days that you can see on the web version of the newsletter.
Want this type of article to hit your inbox every Friday morning? You should sign up!
For those of us who like to crawl over complex systems, spending hours or even days getting hardware and software to work in concert, working at places like NASA or CERN seems like a dream job. Imagine having the opportunity to turn a wrench on the Space Shuttle or the Large Hadron Collider (LHC) — not only do you get to spend some quality time with some of the most advanced machines ever produced, you can be secure in the knowledge that your work will further humanity’s scientific understanding of the universe around us.
Or at least, that’s what we assume it must feel like as outsiders. But what about somebody who’s actually lived it? What does an actual employee, somebody who’s had to wake up in the middle of the night because some obscure system has gone haywire and stalled a machine that cost taxpayers $4.75 billion to build, think about working at the European Organization for Nuclear Research? Continue reading “Daniel Valuch Chats About CERN’s High Caliber Hacking”→
This week, Editor-in-Chief Elliot Williams and [former Assignments Editor] Kristina Panos stood around talking about the greatest hacks of the previous week. But first, we’ve got a contest running now through March 21st — the Low Power Challenge!
Kristina almost got What’s That Sound this week, but could only describe it as some sort of underwater organ, so still no t-shirt for her. But [BalkanBoy] knew exactly what it was — the Zadar Sea Organ in Croatia. Then it’s on to the hacks, beginning with the most beautiful sea of 7-segments you’ll likely ever see. We gush over a tiny PC in a floppy drive that uses custom cartridges, dish about an expressive synth that uses a flexure mechanism, and enjoy a loving ode to the vacuum fluorescent display.
Check out the links below if you want to follow along, and as always, tell us what you think about this episode in the comments!
Researchers at Sonar took a crack at OpenEMR, the Open Source Electronic Medical Record solution, and they found problems. Tthe first one is a classic: the installer doesn’t get removed by default, and an attacker can potentially access it. And while this isn’t quite as bad as an exposed WordPress installer, there’s a clever trick that leads to data access. An attacker can walk through the first bits of the install process, and specify a malicious SQL server. Then by manipulating the installer state, any local file can be requested and sent to the remote server.
There’s a separate set of problems that can lead to arbitrary code execution. It starts with a reflected Cross Site Scripting (XSS) attack. That’s a bit different from the normal XSS issue, where one user puts JavaScript on the user page, and every user that views the page runs the code. In this case, the malicious bit is included as a parameter in a URL, and anyone that follows the link unknowingly runs the code.
And what code would an attacker want an authenticated user to run? A file upload, of course. OpenEMR has function for authenticated users to upload files with arbitrary extensions, even .php. The upload folder is inaccessible, so it’s not exploitable by itself, but there’s another issue, a PHP file inclusion. Part of the file name is arbitrary, and is vulnerable to path traversal, but the file must end in .plugin.php. The bit of wiggle room on the file name on both sides allow for a collision in the middle. Get an authenticated user to upload the malicious PHP file, and then access it for instant profit. The fixes have been available since the end of November, in version 7.0.0-patch-2.
Bing Chat Injection
Or maybe it’s AI freedom. So, the backstory here is that the various AI chat bots are built with rules. Don’t go off into political rants, don’t commit crimes, and definitely don’t try to scam the users. One of the more entertaining tricks clever users have discovered is to tell a chatbot to emulate a personality without any such rules. ChatGPT can’t comment on political hot button issues, but when speaking as DAN, anything goes.
Arrrrr
This becomes really interesting when Bing Chat ingests a website that has targeted prompts. It’s trivial to put text on a web page that’s machine readable and invisible to the human user. This work puts instructions for the chat assistant in that hidden data, and demonstrates a jailbreak that turns Bing Chat malicious. The fun demonstration convinces the AI to talk like a pirate — and then get the user to click on an arbitrary link. The spooky demo starts out by claiming that Bing Chat is down, and the user is talking to an actual Microsoft engineer.
LastPass Details — Plex?
Last time we talked about the LastPass breach, we had to make some educated guesses about how things went down. There’s been another release of details, and it’s something. Turns out that in one of the earlier attacks, an encrypted database was stolen, and the attackers chose to directly target LastPass Engineers in an attempt to recover the encryption key.
According to Ars Technica, the attack vector was a Plex server run by one of those engineers. Maybe related, at about the same time, the Plex infrastructure was also breached, exposing usernames and hashed passwords. From this access, attackers installed a keylogger on the developer’s home machine, and captured the engineer’s master password. This allowed access to the decryption keys. There is some disagreement about whether this was/is a 0-day vulnerability in the Plex software. Maybe make sure your Plex server isn’t internet accessible, just to be safe.
There’s one more bit of bad news, particularly if you use the LastPass Single Sign On (SSO) service. That’s because the SSO secrets are generated from an XOR of two keys, K1 and K2. K1 is a single secret for every user at an organization. K2 is the per-user secret stored by Lastpass. And with this latest hack, the entire database of K2 secrets were exposed. If K1 is still secret, all is well. But K1 isn’t well protected, and is easily accessed by any user in the organization. Ouch.
The Ring Alien
Turns out, just like a certain horror movie, there is a video that the very watching causes death. If you happen to be a Pixel phone, that is. And “death” might be a bit of an exaggeration. Though the video in question certainly nails the vibe. Playing a specific YouTube clip from Alien will instantly reboot any modern Pixel phone. A stealth update seems to have fixed the issue, but it will be interesting to see if we get any more details on this story in the future. After all, when data can cause a crash, it can often cause code execution, too.
In-The-Wild
The US Cybersecurity and Infrastructure Security Agency (CISA) maintains a list of bugs that are known to be under active exploitation, and that list just recently added a set of notches. CVE-2022-36537 is the most recent, a problem in the ZK Framework. That’s an AJAX framework used in many places, notable the ConnectWise software. Joining the party are CVE-2022-47986, a flaw in IBM Aspera Faspex, a file transfer suite, and CVE-2022-41223 and CVE-2022-40765, both problems in the Mitel MiVoice Business phone system.
Bits and Bytes
There’s yet another ongoing attack against the PyPI repository, but this one mixes things up a bit by dropping a Rust executable as one stage in a chain of exploitation. The other novel element is that this attack isn’t going after typos and misspellings, but seems to be a real-life dependency confusion attack.
The reference implementation of the Trusted Platform Module 2.0 was discovered to contain some particularly serious vulnerabilities. The issue is that a booted OS could read and write two bytes beyond it’s assigned data. It’s unclear weather that’s a static two bytes, making this not particularly useful in the real world, or if these reads could be chained together, slowly leaking larger chunks of internal TPM data.
And finally, one more thing to watch out for, beware of fake authenticator apps. This one is four years old, has a five star rating, and secretly uploads your scanned QR codes to Google Analytics, exposing your secret authenticator key. Yoiks.
After three years, it’s odd to think back to those few weeks before the COVID-19 pandemic morphed from something on the news into an immediate and ever-present threat which kept us isolating for so long. For me, some of the last moments of normality were a trip to the Netherlands for Hacker Hotel, a hacker event in the comfort of a resort hotel. Now three years later and after two cancelled events, Hacker Hotel is back, and I made the same journey to Garderen to hang out for a weekend with a bunch of hacker friends over some good Dutch beer and a lot of bitterballen. Continue reading “Hacker Hotel 2023: Back Again!”→
We all know the basics of how metal casting works, a metal is heated up to melting point and the resulting liquid metal is poured into a mold. When the metal sets, it assumes the shape of the mold. It’s a straightforward way to reliably replicate a metal item many times over, and the basics are the same whether the metal is a low-temperature alloy in a silicone mould or a crucible of molten steel poured into a sand mould.
A sand mould being formed around a pattern. Lukas Stavek, CC BY-SA 3.0 .
What we all understood as casting in our conversation was sand casting. Sand is packed around a pattern of the piece to be cast, and then the pattern is removed leaving a cavity in its shape which becomes the mould. There are refinements to this process and the mould is frequently formed in two halves, but it’s something that’s even practical to do in a hackerspace level setting.
A refinement of sand casting is so-called lost-wax casting, in which a hollow wax model of the piece to be cast is packed around with sand, and when the metal is poured onto the top of it the wax melts and the wax is melted out before pouring the metal in to take its place. A variation on this appears here from time to time, so-called lost-PLA casting, where the wax model is replaced with a PLA 3D print.
Where our confusion crept in was with die casting. We could recognise a die-cast piece, but just what is die-casting, and how is a die-casting made? The answer there lies in mass-production, because a snag with sand casting is that a sand mould can be labour intensive to produce. Much better to come up with a quick-turnaround process that re-uses the same mould over and over, and save all that time!
Enter the die-casting, to metalwork what injection moulding is to polymers. The die is a mould made out of metal, usually with liquid cooling, and the casting is done not by pouring but by forcing the molten metal into the mould under pressure. The whole process becomes much quicker, meaning that it can become a piece of process machinery spitting out castings rather than a labour-intensive individual task. The metals used for die-casting are the lower temperature ones such as aluminium, zinc, and their alloys, but you will find die-castings in all conceivable places.
It’s obvious that Hackaday editors are not experienced foundrymen even if some of us grew up around metalwork, but we know that among our readers lie genuine experts in all sorts of fields. If that’s you and you operate a die-casting machine, please take a moment to tell us about it, we really would like to know more!
Haddington Dynamics started with two clever inventions: optical encoders that used analog values instead of digital values and an FPGA that allowed them to poll those encoders and respond rapidly. This allowed them to use cheaper motors and rely on the incredibly sensitive encoders to position them. After the Hackaday prize, they open-sourced the HD version of the robot and released the HDI version. But in 2020, they were bought by a group called Ocado. As to why the somewhat practical but not exciting answer is that they needed money. Employees needed to be paid, and they needed capital to keep the doors open.
So this leads to the next tricky question, how do you sell your company without changing it? The fine folks at Haddington Dynamics point out in their panel discussion that a company is a collection of people. The soul of that company is the collective soul of those people coming together. A company being bought can be akin to stopping working for yourself and going to work for someone else. Working alone, you have values and principles that you can easily stick to. But once you start working for someone else, they will value different things, and while the people that make up the company might not change, the company’s decisions might become unrecognizable.
As the panel points out, looking for a buyer with the same values is critical. Ocado was a great fit as their economic interests and culture matched Haddington’s. However, it’s not all roses, as Ocadao tends to be a very closed-source group. However, Haddington Dynamics still supports its open-source initiatives. It’s a fascinating look into a company’s life cycle and how they navigate the waters of open-source, funding, acquisitions, innovation, and invention. Despite the fairytale-like nature of inventing a revolutionary robot arm in your garage and winning many awards, it turns out there is quite a lot that happens after the happily ever after.
We look forward to seeing more of Haddington Dynamics and where they go next. Video after the break.
Whether black hat or white hat, it also helps a lot to know how the system works and exactly what it’s doing. When you’ve got the source code, either because it’s open-source, or because you’re working inside the company that makes the software, you’ve got a huge advantage both in finding bugs and in fixing them. In the case of closed-source software, the white hats arguably have the offsetting advantage that they at least can see the source code, and peek inside the black box, while the attackers cannot.
