The Right Benchmark For GPT

July 29, 2023 by Elliot Williams 41 Comments

Dan Maloney wanted to design a part for 3D printing. OpenSCAD is a coding language for generating 3D objects. ChatGPT can write code. What could possibly go wrong? You should go read his article because it’s enlightening and hilarious, but the punchline is that it ran afoul of syntax errors, but also gave him enough of a foothold that he could teach himself enough OpenSCAD to get the project done anyway. As with many people who have asked the AI to create some code, Dan finds that it’s not as good as asking someone who knows what they’re doing, but that it’s also better than nothing.

And this is where I start grumbling. When you type your desires into the word-follower machine, your alternative isn’t nothing. Your alternative is to fire up a search engine instead and type “openscad tutorial”. That, for nearly any human endeavor, will get you a few good guides, written by humans who are probably expert in the subject in question, and which are aimed at teaching you the thing that you want to learn. It doesn’t get better than that. You’ll be up and running with your design in no time.

Indeed, if you think about the relevant source material that the LLM was trained on, it’s exactly these tutorials. It can’t possibly do better than the best of them, although the resulting average tutorial might be better than the worst you’ll find. (Some have speculated on what happens when the entire Internet is filled with these generated texts – what will future AIs learn from?)

In Dan’s case, though, he didn’t necessarily want to learn OpenSCAD – he just wanted the latch designed. But in the end, he had to learn enough OpenSCAD to get the AI code compiling without error. He spent an hour learning OpenSCAD and now he’s good to go on his next project too.

So the next time you hear someone say that they got an answer back from a large language model that wasn’t perfect, but it was “better than nothing”, think critically if “nothing” is really the right benchmark.

Do you really want to learn nothing? Do you really have no resources to get started with? I would claim that we have the most amazing set of tutorial resources the world has ever known at our fingertips. Compared to the ability to teach millions of humans to achieve their own goals, that makes the LLM party tricks look kinda weak, in my opinion.

Would We Recognize Extraterrestrial Technology If We Saw It?

July 28, 2023 by Maya Posch 92 Comments

There’s a common critique in science fiction series like Star Trek about the extraterrestrial species not looking ‘alien’ enough, as well as about their technology being strangely similar to our own, not to mention compatible to the point where their widgets can be integrated into terrestrial systems by any plucky engineer. Is this critique justified, or perhaps more succinctly put: if we came across real extraterrestrial life with real extraterrestrial technology, would we even notice? Would an alien widget borrowed of an alien spacecraft even work with our own terrestrial spacecraft’s system?

Within the domain of exobiology there are still plenty of discussions on the possible formation and evolutionary paths conceivable within the Universe, but the overarching consensus seems to be that it’s hard to escape the herding effect of fundamental physics. For lifeforms, carbon-based chemistry is the only reasonable option, and when it comes to technology, it’s hard to not end up at technology using the same physical principles which we presume to exist across the Universe, which would practically guarantee some level of interoperability.

What’s notable here is that over the past years, a number of people have claimed to have observed potential alien technology in our Solar System, in particular the ʻOumuamua asteroid in 2017 and a more recent claim by astrophysicist Abraham Loeb regarding an interstellar meteor that impacted Earth in 2019, which he says could be proof of ‘alien technology’. This raises the question of whether we are literally being pummeled by extraterrestrial spacecraft these days.

Continue reading “Would We Recognize Extraterrestrial Technology If We Saw It?” →

Hackaday Podcast 229: DIY VR, Gutting Voice Assistants, And ChatGPT Failing Its Summer Internship

July 28, 2023 by Kristina Panos 1 Comment

This week, Editor-in-Chief Elliot Williams and Kristina Panos braved the slight cold and the high heat respectively to bring you the best hacks of the previous seven days. In the news this week: you’ve got a second and final chance to get your Supercon talk proposals in! So get on that, because we need YOU to help make Supercon awesome.

We can chalk up a win for Kristina on What’s That Sound this week (finally!). Will you get it right? Will you get it exactly right? Time and Elliot’s fancy dice will tell.

Then it’s on to the hacks, where we check out a cool RFID emulator e-paper badge, discuss whether a certain type of record player is better off as a cyberdeck, and look through the eye of a Gameboy-style camera for the Playdate console.

From there we take a look at gutting and rooting voice assistants, a solenoid keyboard, and a beautifully rebuilt VR headset that now does AR as well. Finally we talk autonomous solar boats, lambast ChatGPT as the worst summer intern ever, and ponder what makes a thing count as Linux or not.

Check out the links below if you want to follow along, and as always, tell us what you think about this episode in the comments!

Download and savor at your leisure.

Continue reading “Hackaday Podcast 229: DIY VR, Gutting Voice Assistants, And ChatGPT Failing Its Summer Internship” →

This Week In Security: Zenbleed, Web Integrity, And More!

July 28, 2023 by Jonathan Bennett 9 Comments

Up first is Zenbleed, a particularly worrying speculative execution bug, that unfortunately happens to be really simple to exploit. It leaks data from function like strlen, memcpy, and strcmp. It’s vulnerable from within virtual machines, and potentially from within the browser. The scope is fairly limited, though, as Zenbleed only affects Zen 2 CPUs: that’s the AMD Epyc 7002 series, the Ryzen 3000 series, and some of the Ryzen 4000, 5000, and 7020 series of CPUs, specifically those with the built-in Radeon graphics.

And at the heart of problem is a pointer use-after-free — that happens inside the CPU itself. We normally think of CPU registers as fixed locations on the silicon. But in the case of XMM and YMM registers, there’s actually a shared store of register space, and the individual registers are mapped into that space using a method very reminiscent of pointers.

Continue reading “This Week In Security: Zenbleed, Web Integrity, And More!” →

Did TETRA Have A Backdoor Hidden In Encrypted Police And Military Radios?

July 27, 2023 by Lewin Day 42 Comments

Encrypted communications are considered vital for many organizations, from military users to law enforcement officers. Meanwhile, the ability to listen in on those communications is of great value to groups like intelligence agencies and criminal operators. Thus exists the constant arms race between those developing encryption and those desperately eager to break it.

In a startling revelation, cybersecurity researchers have found a potentially intentional backdoor in encrypted radios using the TETRA (TErrestrial Trunked RAdio) standard. TETRA equipment is used worldwide by law enforcement agencies, military groups, and critical infrastructure providers, some of which may have been unintentionally airing sensitive conversations for decades.

Continue reading “Did TETRA Have A Backdoor Hidden In Encrypted Police And Military Radios?” →

How To Survive A Wet Bulb Event

July 27, 2023 by Lewin Day 268 Comments

Territories across the northern hemisphere are suffering through record-breaking heatwaves this summer. Climate scientists are publishing graphs with red lines jagging dangerously upwards as unprecedented numbers pour in. Residents of the southern hemisphere watch on, wondering what the coming hot season will bring.

2023 is hinting at a very real climate change that we can’t ignore. As the mercury rises to new heights, it’s time to educate yourself on the very real dangers of a wet bulb event. Scientists predict that these deadly weather conditions could soon strike in the hottest parts of the world. What you learn here could end up saving your life one day.

Hot Bodies

The body has methods of maintaining a set temperature. *Credit: Wikimedia Commons, CNX OpenStax, CC BY-SA 4.0*

To understand the danger of a wet bulb event, we must first understand how our bodies work. The human body likes to maintain its temperature at approximately 37 °C (98.6 °F). That temperature can drift slightly, and the body itself will sometimes move its temperature setpoint higher to tackle infection, for example. The body is a delicate thing, however, and a body temperature above 40 °C (104 °F) can become life threatening. Seizures, organ failures, and unconsciousness are common symptoms of an overheating human. Death is a near-certainty if the body’s temperature reaches 44 °C (112 °F), though in one rare case, a patient in a coma survived a body temperature of 46.5 °C (115.7 °F).

Thankfully, the body has a host of automated systems for maintaining its temperature at its chosen set point. Blood flow can be controlled across the body, and we instinctively seek to shed clothes in the heat and cover ourselves in the cold. However, the bare naked fact is that one system is most crucial to our body’s ability to cool itself. The perspiration system is vital, as it uses sweat to cool our body via evaporation. Water is a hugely effective coolant in this way, with beads of sweat soaking up huge amounts of heat from our skin as they make the phase change from liquid to vapor.

Continue reading “How To Survive A Wet Bulb Event” →

ChatGPT, The Worst Summer Intern Ever

July 26, 2023 by Dan Maloney 71 Comments

Back when I used to work in the pharma industry, I had the opportunity to hire summer interns. This was a long time ago, long enough that the fresh-faced college students who applied for the gig are probably now creeping up to retirement age. The idea, as I understood it, was to get someone to help me with my project, which at the time was standing up a distributed data capture system with a large number of nodes all running custom software that I wrote, reporting back to a central server running more of my code. It was more work than I could manage on my own, so management thought they’d take mercy on me and get me some help.

The experience didn’t turn out quite like I expected. The interns were both great kids, very smart, and I learned a lot from them. But two months is a very tight timeframe, and getting them up to speed took up most of that time. Add in the fact that they were expected to do a presentation on their specific project at the end of the summer, and the whole thing ended up being a lot more work for me than if I had just done the whole project myself.

I thought about my brief experience with interns recently with a project I needed a little help on. It’s nothing that hiring anyone would make sense to do, but still, having someone to outsource specific jobs to would be a blessing, especially now that it’s summer and there’s so much else to do. But this is the future, and the expertise and the combined wisdom of the Internet are but a few keystrokes away, right? Well, maybe, but as you’ll see, even the power of large language models has its limit, and trying to loop ChatGPT in as a low-effort summer intern leaves a lot to be desired.

Continue reading “ChatGPT, The Worst Summer Intern Ever” →