A Raspberry Pi with a camera is nothing new. But the Pixy2 camera can interface with a variety of microcontrollers and has enough smarts to detect objects, follow lines, or even read barcodes without help from the host computer. [DroneBot Workshop] has a review of the device and he’s very enthused about the camera. You can see the video below.
When you watch the video, you might wonder how much this camera will cost. Turns out it is about $60 which isn’t cheap but for the capabilities it offers it isn’t that much, either. The camera can detect lines, intersections, and barcodes plus any objects you want to train it to recognize. The camera also sports its own light source and dual servo motor drive meant for a pan and tilt mounting arrangement.
You can connect via USB, serial, SPI, or I2C. Internally, the camera processes at 60 frames per second and it can remember seven signatures internally. There’s a PC-based configuration program that will run on Windows, Mac, or Linux. You can even use the program to spy on the camera while it is talking to another microcontroller like an Arduino.
The camera isn’t made to take sharp photos or video, but it is optimized for finding things, not for picture quality. High-quality frames take more processing power, so this is a reasonable trade. The camera does need training to find objects by color and shape. You can do the training with the PC-based software, but you can also do it with a self-contained procedure that relies on a button on the camera. The video shows both methods.
Once trained, you can even have an Arduino find objects. There’s a library that allows you to find how many items the camera currently sees and find out what the block is and its location. The identification clearly depends highly on color, so you’ll probably need to experiment if you have things that are different colors on different sides or has multiple colors.
Sure, you could use a sufficient computer with OpenCV to get some of these results, but having this all in one package and usable from just about any processor could be a real game-changer for the right kind of project. If you wanted to make a fancy line-following robot that could handle 5-way intersections and barcode commands this would be a no-brainer.
We’ve seen other smart cameras like OpenMV before. Google also has a vision processor for the Pi, too. It has a lot of capability but assumes you are connecting to a Pi.
$60 is really cheap as compared to the cameras I work with thant can cost up to $200K. FTR cameras and high end mapping cameras can run up to $1M.
Currently I am working with cameras that have embedded computers or FPGA’s for Industrial Machine Vision.
Look at the NVidia TX1 and TX2. Very powerful and not a bad price for a micro computer.
The more direct competition for this would be the JeVois, which is also around $50. So yeah, it’s cheap, and I don’t know how it stacks up to the JeVois, but the paradigm of “Small, onboard CV for under $100” has certainbly already been established.
JeVois is incredibly more capable., and cheaper It can run OpenCV and Tensorflow Lite directly, and being able to run Python it´s very flexible. Plus JeVois has a variety of camera sensors and lenses (fish-eye, low light, noIR…)
On the other hand, JeVois is more energy-hungry, and this damn cooling fan it is a real bummer. Milling an aluminum block to host the electronics and sensors results in a very sturdy, field-ready Machine Vision solution.
Isn’t “a sufficient computer with OpenCV” a raspberry Pi?
I could see how it might be useful with arduino/esp32 etc., but RPi 3 can run OpenCV, can access it with C, Python etc.,, can get a camera for it fairly cheap, not sure what this would add. Maybe its easier to interface to? I suppose the pan/tilt might come in handy.
It seems it would off load the video processing stuff from the pi. While the pi can do xyz with open cv it can’t do much else at the same time.
I would agree. I would think it would be well worth it to offload your main processor even if that processor might be able to do the task.
So then still cheaper and more capable to buy a second PI to handle OpenCV and communicate to the first PI.
If your time is worth nothing or worth very little, then yes. I mean, we all build on abstractions, even the Pi itself.
Seems to me like that recent RISC-V based vision system should be much more powerful…
Oh? Tell me more. I’ve only recently learned of RISC-V and hardware with that ISA that’s available to the DIY market. Does RISC-V lend itself to vision processing better than, say. ARM?
I’ve used one of these cameras out in the wild and the barcode recognition is excellent. Line following, however, requires good lighting conditions, eg big LED lamp shining down on the ground and does not like deep shadows.
I don’t think RPi is a viable alternative. For a start, this Pixy2 can be set up in minutes rather than hours/days. Also, graphics processing on the Pi is relatively poor. The sensible upgrade from Pixy2 would be Nvidia Jetson TX2 which is basically a monster Pi with on-board camera.
I’ll let Hackaday know when I’ve got some (more) meaningful results from Jeston TX2 via the tip line. At present it works very well inferring the position of dogs!
I’ve used the Pixy and Pixy 2 in two different projects for work. One was a semi-autonomous children’s bike that tracked a marker on the back of a lead cyclist, and one was a musical instrument that used colours as sounds.
These things are almost plug and play, and I used one with a Raspberry Pi in both projects. The colour detection is spot on in lots of lighting conditions, though I’ve found it doesn’t like red or non-matte objects. Spheres are better than flat planes, too due to the specular reflections. I totally agree with the setup time being minutes compared with OpenCV (although if someone were to package up blob tracking as a bit of code then it might be similar). One catch is that the Python API is missing some features that the C library offers, and so there’s a little bit extra setup porting those commands.
See licence plate X, open garage door and turn lights on might be nice for $60!
Hey Lee, unfortunately Pixy2 doesn’t recognize letters – just colors – so this wouldn’t be possible!
were you able to implement same ?
On this topic, I would love to figure out how to route all my ip cameras through a middle man that uses something like OpenCV and Tensor flow, sends the video out with the updated data as if the camera had detected it, so things like Synology or Blue Iris can continue to store and alert me me.
I hope this camera had a better quality sensor than the Pixy 1. I had to return it as it could not detect anything unless the lighting and contrast conditions were ideal (i.e., a yellow ball over a flat white background with very good studio lights on). A yellow ball over a wooden desk? Nah.
Hey Casco, sorry you had trouble with original Pixy! Did you ever try adjusting the Signature Tuning sliders in PixyMon? They can help a lot with fine-tuning object detection.
Yes, this is years later that I’m responding to this, and I bought a Pixy2. But in hopes that someone reads this via keyword searches….”Casco” mentions that the lighting conditions need to be ideal. THAT IS AN UNDERSTATEMENT. Unless the target object is very close to the camera, forget it. Yes, I know, it depends upon how close and colorful the object is. Every demo you see online has the target object quite close to the camera. If you move the camera say, 36″ away from a colored golf ball, good luck!
I’m not saying this camera is a gimmic. I’m just saying that when I read a support person in the Pixy2 forum stating that the camera can detect an object a mile away????…yeah…NOT!
Long story short…this camera is only reliable under 100% perfect conditions at an extremely short distance to the object. And when I say extremely, I’m talking (and have experienced), less than a foot.
Hey Al,
Jesse here from Charmed Labs (creators of Pixy2). I wanted to clarify that Pixy2 does not recognize shapes (besides the custom barcodes we provide). Also, the detection algorithm we use relies entirely on hue, aka color. So your statement “The camera does need training to find objects by color and shape.” is misleading.
Thanks for the writeup!
Cheers,
Jesse French
Production Manager
Charmed Labs
Austin, TX
is pixy2 are capable of recognizing different sizes of coins? for image processing??
Seems like a cool and cheap product,wish I had more time to play with it.
Wondering if you have tried a new camera HuskyLens, seems cheaper than pixy and jevois, not sure what’s the difference and how it would work like.