Building AI Models To Diagnose HVAC Issues

August 8, 2024 by Bryan Cockfield 10 Comments

HVAC – heating, ventilation, and air conditioning – can account for a huge amount of energy usage of a building, whether it’s residential or industrial. Often it’s the majority energy consumer, especially in places with extreme climates or for things like data centers where cooling is a large design consideration. When problems arise with these complex systems, they can go undiagnosed for a time and additionally be difficult to fix, leading to even more energy losses until repairs are complete. With the growing availability of platforms that can run capable artificial intelligences, [kutluhan_aktar] is working towards a system that can automatically diagnose potential issues and help humans get a handle on repairs faster.

The prototype system is designed for hydronic (water-based) systems and uses two separate artificial intelligences, one to analyze thermal imagery of the system and look for problems like leaks, hot spots, or blockages, and the other to listen for anomalous sounds especially relating to the behavior of cooling fans. For the first, a CNC-like machine was built to move a thermal camera around a custom-built model HVAC system and report its images back to a central system where they can be analyzed for anomalies. The second system which analyses audio runs its artificial intelligence on a XIAO ESP32C6 and listens to the cooling fans running in the model.

One problem that had to be tackled before any of this could be completed was actually building an open-source dataset to train the AI on. That’s part of the reason for the HVAC model in this project; being able to create problems to train the computer to detect before rolling it out to a larger system. The project’s code and training models can be found on its GitHub page. It seems to be a fairly robust solution to this problem, though, and we’ll be looking forward to future versions running on larger systems. Not everyone has a hydronic HVAC system, though. As heat pumps become more and more popular and capable, you’ll need systems to control those as well.

AI Image Generator Twists In Response To MIDI Dials, In Real-time

July 30, 2024 by Donald Papp 29 Comments

MIDI isn’t just about music, as [Johannes Stelzer] shows by using dials to adjust AI-generated imagery in real-time. The results are wild, with an interactivity to them that we don’t normally see in such things.

[Johannes] uses Stable Diffusion‘s SDXL Turbo to create a baseline image of “photo of a red brick house, blue sky”. The hardware dials act as manual controls for applying different embeddings to this baseline, such as “coral”, “moss”, “fire”, “ice”, “sand”, “rusty steel” and “cookie”.

By adjusting the dials, those embeddings are applied to the base image in varying strengths. The results are generated on the fly and are pretty neat to see, especially since there is no appreciable amount of processing time required.

The MIDI controller is integrated with the help of lunar_tools, a software toolkit on GitHub to facilitate creating interactive exhibits. As for the image end of things, we’ve previously covered how AI image generators work.

Peering Into The Black Box Of Large Language Models

July 3, 2024 by Donald Papp 35 Comments

Large Language Models (LLMs) can produce extremely human-like communication, but their inner workings are something of a mystery. Not a mystery in the sense that we don’t know how an LLM works, but a mystery in the sense that the exact process of turning a particular input into a particular output is something of a black box.

This “black box” trait is common to neural networks in general, and LLMs are very deep neural networks. It is not really possible to explain precisely why a specific input produces a particular output, and not something else.

Why? Because neural networks are neither databases, nor lookup tables. In a neural network, discrete activation of neurons cannot be meaningfully mapped to specific concepts or words. The connections are complex, numerous, and multidimensional to the point that trying to tease out their relationships in any straightforward way simply does not make sense.

Continue reading “Peering Into The Black Box Of Large Language Models” →

Torment Poor Milton With Your Best Pixel Art

June 25, 2024 by Donald Papp 7 Comments

One of the great things about new tech tools is just having fun with them, like embracing your inner trickster god to mess with ‘Milton’, an AI trapped in an empty room.

Milton is trapped in a room is a pixel-art game with a simple premise: use a basic paint interface to add objects to the room, then watch and listen to Milton respond to them. That’s it? That’s it. The code is available on the GitHub repository, but there’s also a link to play it live without any kind of signup or anything. Give it a try if you have a few spare minutes.

Under the hood, the basic loop is to let the user add something to the room, send the picture of the room (with its new contents) off for image recognition, then get Milton’s reaction to it. Milton is equal parts annoyed and jumpy, and his speech and reactions reflect this.

The game is a bit of a concept demo for Open Souls whose “thing” is providing AIs with far more personality and relatable behaviors than one typically expects from large language models. Maybe this is just what’s needed for AI opponents in things like the putting game of Connect Fore! to level up their trash talking.

Hackaday Links: June 16, 2024

June 16, 2024 by Dan Maloney 16 Comments

Attention, slackers — if you do remote work for a financial institution, using a mouse jiggler might not be the best career move. That’s what a dozen people learned this week as they became former employees of Wells Fargo after allegedly being caught “simulating keyboard activity” while working remotely. Having now spent more than twice as many years working either hybrid or fully remote, we get it; sometimes, you’ve just got to step away from the keyboard for a bit. But we’ve never once felt the need to create the “impression of active work” during those absences. Perhaps that’s because we’ve never worked in a regulated environment like financial services.

For our part, we’re curious as to how the bank detected the use of a jiggler. The linked article mentions that regulators recently tightened rules that require employers to treat an employee’s home as a “non-branch location” subject to periodic inspection. More than enough reason to quit, in our opinion, but perhaps they sent someone snooping? More likely, the activity simulators were discovered by technical means. The article contains a helpful tip to avoid powering a jiggler from the computer’s USB, which implies detecting the device over the port. Our guess is that Wells tracks mouse and keyboard activity and compares it against a machine-learning model to look for signs of slacking.

Continue reading “Hackaday Links: June 16, 2024” →

Giant Brains, Or Machines That Think

June 15, 2024 by Elliot Williams 8 Comments

Last week, I stumbled on a marvelous book: “Giant Brains; or, Machines That Think” by Edmund Callis Berkeley. What’s really fun about it is the way it sounds like it could be written just this year – waxing speculatively about the future when machines do our thinking for us. Except it was written in 1949, and the “thinking machines” are early proto-computers that use relays (relays!) for their logic elements. But you need to understand that back then, they could calculate ten times faster than any person, and they would work tirelessly day and night, as long as their motors keep turning and their contacts don’t get corroded.

But once you get past the futuristic speculation, there’s actually a lot of detail about how the then-cutting-edge machines worked. Circuit diagrams of logic units from both the relay computers and the brand-new vacuum tube machines are on display, as are drawings of the tricky bits of purely mechanical computers. There is even a diagram of the mercury delay line, and an explanation of how circulating audio pulses through the medium could be used as a form of memory.

All in all, it’s a wonderful glimpse at the earliest of computers, with enough detail that you could probably build something along those lines with a little moxie and a few thousands of relays. This grounded reality, coupled with the fantastic visions of where computers would be going, make a marvelous accompaniment to a lot of the breathless hype around AI these days. Recommended reading!

AI Kayak Controller Lets The Paddle Show The Way

June 10, 2024 by Dan Maloney 6 Comments

Controlling an e-bike is pretty straightforward. If you want to just let it rip, it’s a no-brainer — or rather, a one-thumber, as a thumb throttle is the way to go. Or, if you’re still looking for a bit of the experience of riding a bike, sensing when the pedals are turning and giving the rider a boost with the motor is a good option.

But what if your e-conveyance is more of the aquatic variety? That’s an interface design problem of a different color, as [Braden Sunwold] has discovered with his DIY e-kayak. We’ve detailed his work on this already, but for a short recap, his goal is to create an electric assist for his inflatable kayak, to give you a boost when you need it without taking away from the experience of kayaking. To that end, he used the motor and propeller from a hydrofoil to provide the needed thrust, while puzzling through the problem of building an unobtrusive yet flexible controller for the motor.

His answer is to mount an inertial measurement unit (IMU) in a waterproof container that can clamp to the kayak paddle. The controller is battery-powered and uses an nRF link to talk to a Raspberry Pi in the kayak’s waterproof electronics box. The sensor also has an LED ring light to provide feedback to the pilot. The controller is set up to support both a manual mode, which just turns on the motor and turns the kayak into a (low) power boat, and an automatic mode, which detects when the pilot is paddling and provides a little thrust in the desired direction of travel.

The video below shows the non-trivial amount of effort [Braden] and his project partner [Jordan] put into making the waterproof enclosure for the controller. The clamp is particularly interesting, especially since it has to keep the sensor properly oriented on the paddle. [Braden] is working on a machine-learning method to analyze paddle motions to discern what the pilot is doing and where the kayak goes. Once he has that model built, it should be time to hit the water and see what this thing can do. We’re eager to see the results.
Continue reading “AI Kayak Controller Lets The Paddle Show The Way” →