image recognition

18 Articles

The map in action, along with a sample of the video feeds.

Hardware Store Marauder’s Map Is Clarkian Magic

December 20, 2025 by 23 Comments

The “Marauder’s Map” is a magical artifact from the Harry Potter franchise. That sort of magic isn’t real, but as Arthur C. Clarke famously pointed out, it doesn’t need to be — we have technology, and we can make our own magic now. Or, rather, [Dave] on the YouTube Channel Dave’s Armoury can make it.

[Dave]’s hardware store might be in a rough neighborhood, since it has 50 cameras’ worth of CCTV coverage. In this case, the stockman’s loss is the hacker’s gain, as [Dave] has talked his way into accessing all of those various camera feeds and is using machine vision to track every single human in the store.

Of course, locating individuals in a video feed is easy — to locate them in space from that feed, one first needs an accurate map. To do that, [Dave] first 3D scans the entire store with a rover. The scan is in full 3D, and it’s no small amount of data. On the rover, a Jetson AGX is required to handle it; on the bench, a beefy HP Z8 Fury workstation crunches the point cloud into a map. Luckily it came with 500 GB of RAM, since just opening the mesh file generated from that point cloud needs 126 GB. That is processed into a simple 2D floor plan. While the workflow is impressive, we can’t help but wonder if there was an easier way. (Maybe a tape measure?)

Once an accurate map has been generated, it turns out NVIDIA already has a turnkey solution for mapping video feeds to a 2D spatial map. When processing so much data — remember, there are 50 camera feeds in the store — it’s not ideal to be passing the image data from RAM to GPU and back again, but luckily NVIDIA’s “Deep Stream” pipeline will do object detection and tracking (including between different video streams) all on the GPU. There’s also pose estimation right in there for more accurate tracking of where a person is standing than just “inside this red box”. With 50 cameras, it’s all a bit much for one card, but luckily [Dave]’s workstation has two GPUs.

Once the coordinates are spat out of the neural networks, it’s relatively simple to put footprints on the map in true Harry Potter fashion. It really is magic, in the Clarkian sense, what you can do if you throw enough computing power at it.

Unfortunately for show-accuracy (or fortunately, if you prefer to avoid gross privacy violations), it doesn’t track every individual by name, but it does demonstrate the possibility with [Dave] and his robot. If you want a map of something… else… maybe check out this backyard project.

Continue reading “Hardware Store Marauder’s Map Is Clarkian Magic” →

A Bird Watching Assistant

November 25, 2025 by 21 Comments

When AI is being touted as the latest tool to replace writers, filmmakers, and other creative talent it can be a bit depressing staring down the barrel of a future dystopia — especially since most LLMs just parrot their training data and aren’t actually creative. But AI can have some legitimate strengths when it’s taken under wing as an assistant rather than an outright replacement.

For example [Aarav] is happy as a lark when birdwatching, but the birds aren’t always around and it can sometimes be a bit of a wild goose chase waiting hours for them to show up. To help him with that he built this machine learning tool to help alert him to the presence of birds.

The small device is based on a Raspberry Pi 5 with an AI hat nested on top, and uses a wide-angle camera to keep an eagle-eyed lookout of a space like a garden or forest. It runs a few scripts in Python leveraging the OpenCV library, which is a widely available machine learning tool that allows users to easily interact with image recognition. When perched to view an outdoor area, it sends out an email notification to the user’s phone when it detects bird activity so that they can join the action swiftly if they happen to be doing other things at the time. The system also logs hourly bird-counts and creates a daily graph, helping users identify peak bird-watching times.

Right now the system can only detect the presence of birds in general, but he hopes to build future versions that can identify birds with more specificity, perhaps down to the species. Identifying birds by vision is certainly one viable way of going about this process, but one of our other favorite bird-watching tools was demonstrated by [Benn Jordan] which uses similar hardware but listens for bird calls rather than looking for the birds with a vision-based system.

Continue reading “A Bird Watching Assistant” →

Image Recognition On 0.35 Watts

September 7, 2025 by 23 Comments

Much of the expense of developing AI models, and much of the recent backlash to said models, stems from the massive amount of power they tend to consume. If you’re willing to sacrifice some ability and accuracy, however, you can get ever-more-decent results from minimal hardware – a tradeoff taken by the Grove Vision AI board, which runs image recognition in near-real time on only 0.35 Watts.

The heart of the board is a WiseEye processor, which combines two ARM Cortex M55 CPUs and an Ethos U55 NPU, which handles AI acceleration. The board connects to a camera module and a host device, such as another microcontroller or a more powerful computer. When the host device sends the signal, the Grove board takes a picture, runs image recognition on it, and sends the results back to the host computer. A library makes signaling over I2C convenient, but in this example [Jaryd] used a UART.

To let it run on such low-power hardware, the image recognition model needs some limits; it can run YOLO8, but it can only recognize one object, runs at a reduced resolution of 192×192, and has to be quantized down to INT8. Within those limits, though, the performance is impressive: 20-30 fps, good accuracy, and as [Jaryd] points out, less power consumption than a single key on a typical RGB-backlit keyboard. If you want another model, there are quite a few available, though apparently of varying quality. If all else fails, you can always train your own.

Continue reading “Image Recognition On 0.35 Watts” →

Multi-Year Doorbell Project

September 17, 2023 by 5 Comments

Camera modules for the Raspberry Pi became available shortly after its release in the early ’10s. Since then there has been about a decade of projects eschewing traditional USB webcams in favor of this more affordable, versatile option. Despite the amount of time available there are still some hurdles to overcome, and [Esser50k] has some supporting software to drive a smart doorbell which helps to solve some of them.

One of the major obstacles to using the Pi camera module is that it can only be used by one process at a time. The PiChameleon software that [Esser50k] built is a clever workaround for this, which runs the camera as a service and allows for more flexibility in using the camera. He uses it in the latest iteration of a smart doorbell and intercom system, which uses a Pi Zero in the outdoor unit armed with motion detection to alert him to visitors, and another Raspberry Pi inside with a touch screen that serves as an interface for the whole system.

The entire build process over the past few years was rife with learning opportunities, including technical design problems as well as experiencing plenty of user errors that caused failures as well. Some extra features have been added to this that enhance the experience as well, such as automatically talking to strangers passing by. There are other unique ways of using machine learning on doorbells too, like this one that listens for a traditional doorbell sound and then alerts its user.

Continue reading “Multi-Year Doorbell Project” →

Automated Drone Takes Care Of Weeds

August 10, 2023 by 38 Comments

Commercial industrial agriculture is responsible for providing food to the world’s population at an incredibly low cost, especially when compared to most of human history when most or a majority of people would have been involved in agriculture. Now it’s a tiny fraction of humans that need to grow food, while the rest can spend their time in cities and towns largely divorced from needing to produce their own food to survive. But industrial agriculture isn’t without its downsides. Providing inexpensive food to the masses often involves farming practices that are damaging to the environment, whether that’s spreading huge amounts of synthetic, non-renewable fertilizers or blanket spraying crops with pesticides and herbicides. [NathanBuildsDIY] is tackling the latter problem, using an automated drone system to systemically target weeds to reduce his herbicide use.

The specific issue that [NathanBuildsDIY] is faced with is an invasive blackberry that is taking over one of his fields. To take care of this issue, he set up a drone with a camera and image recognition software which can autonomously fly over the field thanks to Ardupilot and a LiDAR system, differentiate the blackberry weeds from other non-harmful plants, and give them a spray of herbicide. Since drones can’t fly indefinitely, he’s also build an automated landing pad complete with a battery swap and recharge station, which allows the drone to fly essentially until it is turned off and uses a minimum of herbicide in the process.

The entire setup, including drone and landing pad, was purchased for less than $2000 and largely open-source, which makes it accessible for even small-scale farmers. A depressing trend in farming is that the tools to make the work profitable are often only attainable for the largest, most corporate of farms. But a system like this is much more feasible for those working on a smaller scale and the automation easily frees up time that the farmer can use for other work. There are other ways of automating farm work besides using drones, though. Take a look at this open-source robotics platform that drives its way around the farm instead of flying.

Thanks to [PuceBaboon] for the tip!

Continue reading “Automated Drone Takes Care Of Weeds” →

Insect class-order-family-genus-species chart with drawn examples

Neural Network Identifies Insects, Outperforming Humans

July 16, 2022 by 15 Comments

There are about one million known species of insects – more than for any other group of living organisms. If you need to determine which species an insect belongs to, things get complicated quick. In fact, for distinguishing between certain kinds of species, you might need a well-trained expert in that species, and experts’ time is often better spent on something else. This is where CNNs (convolutional neural networks) come in nowadays, and this paper describes a CNN doing just as well if not better than human experts.

Continue reading “Neural Network Identifies Insects, Outperforming Humans” →

A man performing push-ups in front of a PC

Machine Learning Helps You Get In Shape While Working A Desk Job

April 22, 2022 by 9 Comments

Humans weren’t made to sit in front of a computer all day, yet for many of us that’s how we spend a large part of our lives. Of course we all know that it’s important to get up and move around every now and then to stretch our muscles and get our blood flowing, but it’s easy to forget if you’re working towards a deadline. [Victor Sonck] thought he needed some reminders — as well as some not-so-gentle nudging — to get into the habit of doing a quick workout a few times a day.

To this end, he designed a piece of software that would lock his computer’s screen and only unlock it if he performed five push-ups. Locking the screen on his Linux box was as easy as sending a command through the network, but recognizing push-ups was a harder task for which [Victor] decided to employ machine learning. A Raspberry Pi with a webcam attached could do the trick, but the limited processing power of the Pi’s CPU might prove insufficient for processing lots of raw image data.

[Victor] therefore decided on using a Luxonis OAK-1, which is a 4K camera with a built-in machine-learning processor. It can run various kinds of image recognition systems including Blazepose, a pre-trained model that can recognize a person’s pose from an image. The OAK-1 uses this to send out a set of coordinates that describe the position of a person’s head, torso and limbs to the Raspberry Pi through a USB interface. A second machine-learning model running on the Pi then analyzes this dataset to recognize push-ups.

[Victor]’s video (embedded below) is an entertaining introduction into the world of machine-learning systems for video processing, as well as a good hands-on example of a project that results in a useful tool. If you’re interested in learning more about machine learning on small platforms, check out this 2020 Remoticon talk on machine learning on microcontrollers, or this 2019 Supercon talk about implementing machine vision on a Raspberry Pi.

Continue reading “Machine Learning Helps You Get In Shape While Working A Desk Job” →