Pixy2 is Super Vision for Arduino or Raspberry Pi

A Raspberry Pi with a camera is nothing new. But the Pixy2 camera can interface with a variety of microcontrollers and has enough smarts to detect objects, follow lines, or even read barcodes without help from the host computer. [DroneBot Workshop] has a review of the device and he’s very enthused about the camera. You can see the video below.

When you watch the video, you might wonder how much this camera will cost. Turns out it is about $60 which isn’t cheap but for the capabilities it offers it isn’t that much, either. The camera can detect lines, intersections, and barcodes plus any objects you want to train it to recognize. The camera also sports its own light source and dual servo motor drive meant for a pan and tilt mounting arrangement.

You can connect via USB, serial, SPI, or I2C. Internally, the camera processes at 60 frames per second and it can remember seven signatures internally. There’s a PC-based configuration program that will run on Windows, Mac, or Linux. You can even use the program to spy on the camera while it is talking to another microcontroller like an Arduino.

The camera isn’t made to take sharp photos or video, but it is optimized for finding things, not for picture quality. High-quality frames take more processing power, so this is a reasonable trade. The camera does need training to find objects by color and shape. You can do the training with the PC-based software, but you can also do it with a self-contained procedure that relies on a button on the camera. The video shows both methods.

Once trained, you can even have an Arduino find objects. There’s a library that allows you to find how many items the camera currently sees and find out what the block is and its location. The identification clearly depends highly on color, so you’ll probably need to experiment if you have things that are different colors on different sides or has multiple colors.

Sure, you could use a sufficient computer with OpenCV to get some of these results, but having this all in one package and usable from just about any processor could be a real game-changer for the right kind of project. If you wanted to make a fancy line-following robot that could handle 5-way intersections and barcode commands this would be a no-brainer.

We’ve seen other smart cameras like OpenMV before. Google also has a vision processor for the Pi, too. It has a lot of capability but assumes you are connecting to a Pi.

20 thoughts on “Pixy2 is Super Vision for Arduino or Raspberry Pi

  1. $60 is really cheap as compared to the cameras I work with thant can cost up to $200K. FTR cameras and high end mapping cameras can run up to $1M.

    Currently I am working with cameras that have embedded computers or FPGA’s for Industrial Machine Vision.

    Look at the NVidia TX1 and TX2. Very powerful and not a bad price for a micro computer.

    1. The more direct competition for this would be the JeVois, which is also around $50. So yeah, it’s cheap, and I don’t know how it stacks up to the JeVois, but the paradigm of “Small, onboard CV for under $100” has certainbly already been established.

      1. JeVois is incredibly more capable., and cheaper It can run OpenCV and Tensorflow Lite directly, and being able to run Python it´s very flexible. Plus JeVois has a variety of camera sensors and lenses (fish-eye, low light, noIR…)
        On the other hand, JeVois is more energy-hungry, and this damn cooling fan it is a real bummer. Milling an aluminum block to host the electronics and sensors results in a very sturdy, field-ready Machine Vision solution.

  2. I could see how it might be useful with arduino/esp32 etc., but RPi 3 can run OpenCV, can access it with C, Python etc.,, can get a camera for it fairly cheap, not sure what this would add. Maybe its easier to interface to? I suppose the pan/tilt might come in handy.

    1. Oh? Tell me more. I’ve only recently learned of RISC-V and hardware with that ISA that’s available to the DIY market. Does RISC-V lend itself to vision processing better than, say. ARM?

  3. I’ve used one of these cameras out in the wild and the barcode recognition is excellent. Line following, however, requires good lighting conditions, eg big LED lamp shining down on the ground and does not like deep shadows.

    I don’t think RPi is a viable alternative. For a start, this Pixy2 can be set up in minutes rather than hours/days. Also, graphics processing on the Pi is relatively poor. The sensible upgrade from Pixy2 would be Nvidia Jetson TX2 which is basically a monster Pi with on-board camera.

    I’ll let Hackaday know when I’ve got some (more) meaningful results from Jeston TX2 via the tip line. At present it works very well inferring the position of dogs!

    1. I’ve used the Pixy and Pixy 2 in two different projects for work. One was a semi-autonomous children’s bike that tracked a marker on the back of a lead cyclist, and one was a musical instrument that used colours as sounds.

      These things are almost plug and play, and I used one with a Raspberry Pi in both projects. The colour detection is spot on in lots of lighting conditions, though I’ve found it doesn’t like red or non-matte objects. Spheres are better than flat planes, too due to the specular reflections. I totally agree with the setup time being minutes compared with OpenCV (although if someone were to package up blob tracking as a bit of code then it might be similar). One catch is that the Python API is missing some features that the C library offers, and so there’s a little bit extra setup porting those commands.

  4. On this topic, I would love to figure out how to route all my ip cameras through a middle man that uses something like OpenCV and Tensor flow, sends the video out with the updated data as if the camera had detected it, so things like Synology or Blue Iris can continue to store and alert me me.

  5. I hope this camera had a better quality sensor than the Pixy 1. I had to return it as it could not detect anything unless the lighting and contrast conditions were ideal (i.e., a yellow ball over a flat white background with very good studio lights on). A yellow ball over a wooden desk? Nah.

  6. Hey Al,

    Jesse here from Charmed Labs (creators of Pixy2). I wanted to clarify that Pixy2 does not recognize shapes (besides the custom barcodes we provide). Also, the detection algorithm we use relies entirely on hue, aka color. So your statement “The camera does need training to find objects by color and shape.” is misleading.

    Thanks for the writeup!

    Cheers,
    Jesse French
    Production Manager
    Charmed Labs
    Austin, TX

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.