Vizy “AI Camera” Wants To Make Machine Vision Less Complex

October 4, 2020 by Donald Papp 9 Comments

Vizy, a new machine vision camera from Charmed Labs, has blown through their crowdfunding goal on the promise of making machine vision projects both easier and simpler to deploy. The camera, which starts around $250, integrates a Raspberry Pi 4 with built-in power and shutdown management, and comes with a variety of pre-installed applications so one can dive right in.

The Sony IMX477 camera sensor is the same one found in the Raspberry Pi high quality camera, and supports capture rates of up to 300 frames per second (under the right conditions, anyway.) Unlike the usual situation faced by most people when a Raspberry Pi is involved, there’s no need to worry about adding a real-time clock, enclosure, or ensuring shutdowns happen properly; it’s all taken care of.

‘Birdfeeder’ application can automatically identify and upload images of visitors.

Charmed Labs are the same folks behind the Pixy and Pixy 2 cameras, and Vizy goes further in the sense that everything required for a machine vision project has been put onboard and made easy to use and deploy, even the vision processing functions work locally and have no need for a wireless data connection (though one is needed for things like automatic uploading or sharing.) For outdoor or remote applications, there’s a weatherproof enclosure option, and wireless connectivity in areas with no WiFi can be obtained by plugging in a USB cellular modem.

A few of the more hacker-friendly hardware features are things like a high-current I/O header and support for both C/CS and M12 lenses for maximum flexibility. The IR filter can also be enabled or disabled via software, so no more swapping camera modules for ones with the IR filter removed. On the software side, applications are all written in Python and use open software like Tensorflow and OpenCV for processing.

The feature list looks good, but Vizy also seems to have a clear focus. It looks best aimed at enabling projects with the following structure:

Detect Things (people, animals, cars, text, insects, and more) and/or Measure Things (size, speed, duration, color, count, angle, brightness, etc.)

Perform an Action (for example, push a notification or enable a high-current I/O) and/or Record (save images, video, or other data locally or remotely.)

The Motionscope application tracking balls on a pool table. (Click to enlarge)

A good example of this structure is the Birdfeeder application which comes pre-installed. With the camera pointed toward a birdfeeder, animals coming for a snack are detected. If the visitor is a bird, Vizy identifies the species and uploads an image. If the animal is not a bird (for example, a squirrel) then Vizy can detect that as well and, using the I/O header, could briefly turn on a sprinkler to repel the hungry party-crasher. A sample Birdfeeder photo stream is here on Google Photos.

Motionscope is a more unusual but very interesting-looking application, and its purpose is to capture moving objects and measure the position, velocity, and acceleration of each. A picture does a far better job of explaining what Motionscope does, so here is a screenshot of the results of watching some billiard balls and showing what it can do.

Vizy The AI Camera Aims To Ease Machine Vision

August 17, 2020 by Donald Papp 17 Comments

Cameras are getting smarter and more capable than ever, able to run embedded machine vision algorithms and pull off tricks far beyond what something like a serial camera and microcontroller board would be capable of, and the upcoming Vizy aims to be even smarter and easier to use yet. Vizy is the work of Charmed Labs, and this isn’t their first foray into accessible machine vision. Charmed Labs are the same folks behind the Pixy and Pixy 2 cameras. Vizy’s main goal is to make object detection and classification easy, with thoughtful hardware features and a browser-based interface.

Vizy can identify common birds with “Birdfeeder”, one of the several built-in applications that uses local processing only.

The usual way to do machine vision is to get a USB camera and run something like OpenCV on a desktop machine to handle the processing. But Vizy leverages a Raspberry Pi 4 to provide a tightly-integrated unit in a small package with a variety of ready-to-run applications. For example, the “Birdfeeder” application comes ready to take snapshots of and identify common species of bird, while also identifying party-crashers like squirrels.

The demonstration video on their page shows off using the built-in high-current I/O header to control a sprinkler, repelling non-bird intruders with a splash of water while uploading pictures and video clips. The hardware design also looks well thought out; not only is there a safe shutdown and low-power mode for the Raspberry Pi-based hardware, but the lens can be swapped and the camera unit itself even contains an electrically-switched IR filter.

Vizy has a Kickstarter campaign planned, but like many others, Charmed Labs is still adjusting to the changes the COVID-19 pandemic has brought. You can sign up to be notified when Vizy launches; we know we’ll be keen for a closer look once it does. Easier machine vision is always a good thing, because it helps free people to focus on clever ideas like machine vision-based tool alignment.

OAK Vision Modules Help You See The Forest And The Trees

August 9, 2020 by Roger Cheng 13 Comments

OpenCV is an open source library of computer vision algorithms, its power and flexibility made many machine vision projects possible. But even with code highly optimized for maximum performance, we always wish for more. Which is why our ears perk up whenever we hear about a hardware accelerated vision module, and the latest buzz is coming out of the OpenCV AI Kit (OAK) Kickstarter campaign.

There are two vision modules launched with this campaign. The OAK-1 with a single color camera for two dimensional vision applications, and the OAK-D which adds stereo cameras for that third dimension. The onboard brain is a Movidius Myriad X processor which, according to team members who have dug through its datasheet, have been massively underutilized in other products. They believe OAK modules will help the chip fulfill its potential for vision applications, delivering high performance while consuming low power in a small form factor. Reading over the spec sheet, we think it’s fair to call these “Ultimate Myriad X Dev Boards” but we must concede “OpenCV AI Kit” sounds better. It does not provide hardware acceleration for the entire OpenCV library (likely an impossible task) but it does cover the highly demanding subset suitable for Myriad X acceleration.

Since the campaign launched a few weeks ago, some additional information have been released to help assure backers that this project has real substance. It turns out OAK is an evolution of a project we’ve covered almost exactly one year ago that became a real product DepthAI, so at least this is not their first rodeo. It is also encouraging that their invitation to the open hardware community has already borne fruit. Check out this thread discussing OAK for robot vision, where a question was met with an honest “we don’t have expertise there” from the OAK team, but then ArduCam pitched in with their camera module experience to help.

We wish them success for their planned December 2020 delivery. They have already far surpassed their funding goals, they’ve shipped hardware before, and we see a good start to a development community. We look forward to the OAK-1 and OAK-D joining the ranks of other hacking friendly vision modules like OpenMV, JeVois, StereoPi, and AIY Vision.

Dial In Your Multi-Headed 3D Printer With 2020 Machine Vision

June 14, 2020 by Sonya Vasquez 7 Comments

Most folks that have been poking around at multi-tool 3D printing know that lining up nozzles can be a gnarly, but necessary pain point. Existing methods either have us measure offsets with a vernier scale or with a series of pictures taken with an upwards-facing camera. And this step is not to be ignored! Any mismatch between nozzles, and your multicolor prints end up looking like Scotty really screwed up those sliders on that transporter beam console. Fear not, however! [Danal] took this problem as an opportunity to write something that’s completely automated and brought to you by some machine vision.

Dubbed TAMV, for Tool Align Machine Vision, [Danal] added a Raspberry Pi alongside his existing 3D printing motion controller in addition to an upwards facing camera. A few lines of code (and a few hours of compiling OpenCV) later, and he had himself a circle-detecting script that automatically cycles through each tool, detects the nozzle center, and calculates an offset for each tool that’s stored into the machine’s configuration file. If that’s not nifty enough, he’s made the entire setup open-source, and he included both an installation script for compiling OpenCV and a well-written set of step-by-step instructions.

In a world where most hobbyists approaches still solve this problem manually, this is leaps and bounds ahead of what we know, and it’s a great application of machine vision built on top of a stack of recognizable hardware and software. While this project was outfitted for a Jubilee running a Duet3 controller with a Raspberry Pi connected in “single-board computer” mode, the core features are readily adaptable to any other multi-tool machine with a similar control board stack. And for folks willing to poke under the hood, the project could even be extended to a standalone script that you can run on your PC locally to simply print the tool offsets separately.

Alongside TAMV, it’s refreshing that even a decade after 3D printers have been with us, we’re still finding ways to make these machines more capable. For more fresh hacks in this category, check out a new spin on using sharpie ink as a support material release agent.

Sadly, [Danal] has recently passed away in the last week, but we are grateful to capture a snapshot in the history of this person’s life.

Continue reading “Dial In Your Multi-Headed 3D Printer With 2020 Machine Vision” →

Autonomous Sentry Gun Packs A Punch And A Ton Of Build Tips

June 10, 2020 by Dan Maloney 17 Comments

What has dual compressed-air cannons, 500 roll-on deodorant balls, and a machine-learning brain with a bad attitude? We didn’t know either, until [Leo Fernekes] dropped this video on his autonomous robot sentry gun and saw it in action for ourselves.

Now, we’ve seen tons of sentry guns on these pages before, shooting everything from water to various forms of Nerf. And plenty of those builds have used some form of machine vision to aim the gun onto the target. So while it might appear that [Leo]’s plowing old ground here, this build is chock full of interesting tips and tricks.

It started when [Leo] saw a video on TensorFlow basics from our friend [Edje Electronics], which gave him the boost needed to jump into an AI project. The controller he ended up with looks for humans in the scene and slews the turret onto target, where the air cannons can do their thing. The hefty ammo is propelled by compressed air, which is dumped into the chamber using a solenoid valve with an interesting driver that maximizes the speed at which it opens. Style points go to the bacteriophage T4-inspired design, and to the sequence starting at 1:34 which reminded us of the factory scene from RoboCop.

[Leo] really put a ton of work into this project, and the results show. He is hoping to get an art gallery or museum to show it as an interactive piece to comment on one possible robot-human future, presumably after getting guests to sign a release. Whatever happens to it, the robot looks great and [Leo] learned a lot from it, as did we.

Continue reading “Autonomous Sentry Gun Packs A Punch And A Ton Of Build Tips” →

Using Smartphone Cameras To Make Sure Drivers Are Looking At The Road

May 3, 2020 by Maya Posch 25 Comments

Most of us are probably quite aware of the damage that a car can inflict when driven by a distracted driver. In an ideal world, people who are driving a car would not allow something like their phone to distract them from their primary task of being the primary navigation system for the 1+ metric ton vehicle which they are controlling.

Many smartphone apps as well as in-car infotainment systems have added features over the years that try to prevent a driver from using them, but they run into the issue that it’s hard to distinguish between passenger and driver. As it turns out, asking the human driver whether they are the driver doesn’t always get the expected result. This is where [Rushil Khurana] and his team at Carnegie Mellon University (CMU) have come up with a more fool-proof approach.

In their paper (PDF), they cover the algorithm and software implementation that uses the smartphone’s own front (selfie) and back cameras to determine from the car’s interior which side of the car the user is sitting in, and deducing from that whether the user is sitting in the driver’s seat or not. From there it is a fairly safe assumption to make that if the user is sitting in the driver’s seat, and the car is moving, that this user should not be looking at the phone’s screen.

In a test involving 16 different cars and 33 users, they achieved an overall accuracy of 94% with the phone held in the hand, and 92.2% while docked. This is more reliable than the other approaches covered in the paper, and as a benefit does not require any extra hardware. Who knows, upcoming smartphones may include a feature like this, so that apps can easily determine what feature set should be made available to a driver, if any.

Continue reading “Using Smartphone Cameras To Make Sure Drivers Are Looking At The Road” →

Machine Learning Algorithm Runs On A Breadboard 6502

April 22, 2020 by Dan Maloney 7 Comments

When it comes to machine learning algorithms, one’s thoughts do not naturally flow to the 6502, the processor that powered some of the machines in the first wave of the PC revolution. And one definitely does not think of gesture recognition running on a homebrew breadboard version of a 6502 machine, and yet that’s exactly what [Nick Bild] has accomplished.

Before anyone gets too worked up in the comments, we realize that [Nick]’s Vectron breadboard computer is getting a lot of help from other, more modern machines. He’s got a pair of Raspberry Pi 3s in the mix, one to capture and downscale images from a Pi cam, and one that interfaces to an Atari 2600 emulator and sends keypresses to control games based on the gestures seen by the camera. But the logic to convert gesture to control signals is all Vectron, and uses a k-nearest neighbor algorithm executed in 6502 assembly. Fifty gesture images are stored in ROM and act as references for the four known gesture classes: up, down, left, and right. When a match between the camera image and a gesture class is found, the corresponding keypress is sent to the game. The video below shows that the whole thing is pretty responsive.

In our original article on [Nick]’s Vectron breadboard computer, [Tom Nardi] said that “You won’t be playing Prince of Persia on it.” That may be true, but a machine learning system running on the Vectron is not too shabby either.

Continue reading “Machine Learning Algorithm Runs On A Breadboard 6502” →