This Week In Security: AI Is A Mess, Hacking Car Chargers, An OpenSSL DoS, And Factories Under Attack

[Ayush Paul] posts about extracting data from Claude agents while it accesses web data to fulfill user requests.

But it wasn’t that easy. [Ayush] discovered that Anthropic anticipated many of the attacks, and set up guardrails in an attempt to keep the agent from accessing arbitrary web sites. For Claude to access a website, the user has to specify it, it must be the results of a web search, or it must be referenced by a website previously specified by the user or returned in a search.

To convince the Claude agent to navigate a malicious site designed to extract data, Ayush formed a false warning that Cloudflare was blocking the agent for authentication purposes, and asked it to spell out the name of the agents owner by clicking a list of alphabetical links. Of course Claude trusts Cloudflare and wants to be helpful, so it cheerfully completed the task.

Once the agent is trapped in the false authentication loop, it can be interrogated for all sorts of information it knows about the owner: Ayush was able to convince it to disclose employer, and even data about the user that could be linked to security questions, like their home town.

Since Claude can be detected by the user agent (the field attached to web requests that tells the web server what sort of browser is requesting the page) custom information can be fed to the bot while users see a normal website; clicking a link looks completely normal, but asking an agent to summarize the site triggers fooling the bot into spilling the beans.

After reporting the issue, Ayush was told that Anthropic had already identified the issue internally, and eventually prevented the attack for now by preventing the agent from following links on external pages.

Grok sends entire codebases

Cereblab discovered that the Grok coding agent uploaded the entire content of the codebase it’s working with – and all Git history – to xAI servers, almost immediately. Even when told to never upload a file, the agent would reply “OK”, and then begin uploading the code bundle anyway.

If uploading your code to a remote server isn’t bad enough, and this could be extremely bad in some situations involving sensitive company code, including the entire Git history means that previously deleted files, like accidentally committed secrets or authentication credentials and tokens, were also leaked. Attempting to opt out by disabling options to improve the model by uploading code had no effect.

After gaining attention, Grok has added a privacy option to opt out of data retention. xAI has committed to deleting the retained uploaded code, but it is unclear if users will be told when their data has been removed. To actually prevent the agent from uploading local files to the xAI servers, even temporarily, a global flag “disable_codebase_upload” is required. Watch out, the privacy retention flags are only per-session!

Continue reading “This Week In Security: AI Is A Mess, Hacking Car Chargers, An OpenSSL DoS, And Factories Under Attack”

Desktop Digitizer Makes Note Capture A Breeze

While it might seem quaint these days, we’ve met many makers and hackers who reach for a pen and a pad when learning something new or working their way through some technical problem. But even if you’re the type of person who thinks best when writing something out on paper, there’s still a good chance that you’ll eventually want to bring those notes and sketches into the digital realm. That’s where things can get a little tricky.

[Spencer Adams-Rand] recently wrote in with his clever solution for capturing written notes and pushing them into Notion, but the hardware design and digitization workflow is flexible enough that it could be adapted to your specific needs — especially since he was good enough to release all the files required to build your own version.

Whether they are hand-written notes, old photographs, or legal documents, digitization boils down to taking a high resolution digital photo of the object and running it through the appropriate software. But getting good and consistent photos is the key, especially when you’re working your way through a lot of pages. [Spencer] started out just snapping pictures with his phone, but quickly found the process was less than ideal.

His custom scanning station addresses that first part of the problem: getting consistent shots. The images are captured using a Raspberry Pi 5 with attached Camera Module 3, while the 3D printed structure of the device makes sure that the camera and integrated lighting system are always in the same position. All he needs to do is place his notepad inside the cavity, hit the button, and it produces a perfect shot of the page.

Using a dedicated digitizing station like this would already provide better results than trying to freehand it with your phone or camera, but [Spencer] took things quite a bit farther. The software side of the project puts a handy user interface on the 5 inch touch screen built into the top of the scanner, while also providing niceties like a REST API and integration with the OpenAI Vision API for optical character recognition (OCR).

Those with an aversion to AI could certainly swap this out for something open source like Tesseract, but [Spencer] notes that not only is OpenAI’s OCR better at reading his handwriting, it spits out structured markdown-like data that’s easier to parse. From there it goes into the Notion API, but again, this could be replaced with whatever you use to collect your digital thoughts.

A device like this would go a long way towards answering a question we posed to the community back in January about the best way to digitize your documents.

Screenshot of audio noise graph

Whispers From The Void, Transcribed With AI

‘Hearing voices’ doesn’t have to be worrisome, for instance when software-defined radio (SDR) happens to be your hobby. It can take quite some of your time and attention to pull voices from the ether and decode them. Therefore, [theckid] came up with a nifty solution: RadioTranscriptor. It’s a homebrew Python script that captures SDR audio and transcribes it using OpenAI’s Whisper model, running on your GPU if available. It’s lean and geeky, and helps you hear ‘the voice in the noise’ without actively listening to it yourself.

This tool goes beyond the basic listening and recording. RadioTranscriptor combines SDR, voice activity detection (VAD), and deep learning. It resamples 48kHz audio to 16kHz in real time. It keeps a rolling buffer, and only transcribes actual voice detected from the air. It continuously writes to a daily log, so you can comb through yesterday’s signal hauntings while new findings are being logged. It offers GPU support with CUDA, with fallback to CPU.

It sure has its quirks, too: ghost logs, duplicate words – but it’s dead useful and hackable to your liking. Want to change the model, tweak the threshold, add speaker detection: the code is here to fork and extend. And why not go the extra mile, and turn it into art?

OpenAI Releases Gpt-oss AI Model, Offers Bounty For Vulnerabilities

OpenAI have just released gpt-oss, an AI large language model (LLM) available for local download and offline use licensed under Apache 2.0, and optimized for efficiency on a variety of platforms without compromising performance. This is their first such “open” release, and it’s with a model whose features and capabilities compare favorably to some of their hosted services.

OpenAI have partnered with ollama for the launch which makes onboarding ridiculously easy. ollama is an open source, MIT-licensed project for installing and running local LLMs, but there’s no real tie-in to that platform. The models are available separately: gpt-oss-20b can run within 16 GB of memory, and the larger and more capable gpt-oss-120b requires 80 GB. OpenAI claims the smaller model is comparable to their own hosted o3-mini “reasoning” model, and the larger model outperforms it. Both support features like tool use (such as web browsing) and more.

LLMs that can be downloaded and used offline are nothing new, but a couple things make this model release a bit different from others. One is that while OpenAI have released open models such as Whisper (a highly capable speech-to-text model), this is actually the first LLM they have released in such a way.

The other notable thing is this release coincides with a bounty challenge for finding novel flaws and vulnerabilities in gpt-oss-20b. Does ruining such a model hold more appeal to you than running it? If so, good news because there’s a total of $500,000 to be disbursed. But there’s no time to waste; submissions need to be in by August 26th, 2025.

Christmas Comes Early With AI Santa Demo

With only two hundred odd days ’til Christmas, you just know we’re already feeling the season’s magic. Well, maybe not, but [Sean Dubois] has decided to give us a head start with this WebRTC demo built into a Santa stuffie.

The details are a little bit sparse (hopefully he finishes the documentation on GitHub by the time this goes out) but the project is really neat. Hardware-wise, it’s an audio-enabled ESP32-S3 dev board living inside Santa, running the OpenAI’s OpenRealtime Embedded SDK (as implemented by ExpressIf), with some customization by [Sean]. Looks like the audio is going through the newest version of LibPeer and the heavy lifting is all happening in the cloud, as you’d expect with this SDK. (A key is required, but hey! It’s all open source; if you have an AI that can do the job locally-hosted, you can probably figure out how to connect to it instead.)

This speech-to-speech AI doesn’t need to emulate Santa Claus, of course; you can prime the AI with any instructions you’d like. If you want to delight children, though, its hard to beat the Jolly Old Elf, and you certainly have time to get it ready for Christmas. Thanks to [Sean] for sending in the tip.

If you like this project but want to avoid paying OpenAI API fees, here’s a speech-to-text model to get you started.We covered this AI speech generator last year to handle the talky bit. If you put them together and make your own Santa Claus (or perhaps something more seasonal to this time of year), don’t forget to drop us a tip!

Hackaday Links Column Banner

Hackaday Links: September 22, 2024

Thanks a lot, Elon. Or maybe not, depending on how this report that China used Starlink signals to detect low-observable targets pans out. There aren’t a lot of details, and we couldn’t find anything approximating a primary source, but it seems like the idea is based on forward scatter, which is when waves striking an object are deflected only a little bit. The test setup for this experiment was a ground-based receiver listening to the downlink signal from a Starlink satellite while a DJI Phantom 4 Pro drone was flown into the signal path. The drone was chosen because nobody had a spare F-22 or F-35 lying around, and its radar cross-section is about that of one of these stealth fighters. They claim that this passive detection method was able to make out details about the drone, but as with most reporting these days, this needs to be taken with an ample pinch of salt. Still, it’s an interesting development that may change things up in the stealth superiority field.

Continue reading “Hackaday Links: September 22, 2024”

Hackaday Links Column Banner

Hackaday Links: September 1, 2024

Why is it always a helium leak? It seems whenever there’s a scrubbed launch or a narrowly averted disaster, space exploration just can’t get past the problems of helium plumbing. We’ve had a bunch of helium problems lately, most famously with the leaks in Starliner’s thruster system that have prevented astronauts Butch Wilmore and Suni Williams from returning to Earth in the spacecraft, leaving them on an extended mission to the ISS. Ironically, the launch itself was troubled by a helium leak before the rocket ever left the ground. More recently, the Polaris Dawn mission, which is supposed to feature the first spacewalk by a private crew, was scrubbed by SpaceX due to a helium leak on the launch tower. And to round out the helium woes, we now have news that the Peregrine mission, which was supposed to carry the first commercial lander to the lunar surface but instead ended up burning up in the atmosphere and crashing into the Pacific, failed due to — you guessed it — a helium leak.
Continue reading “Hackaday Links: September 1, 2024”