A Fast And Easy-to-use Vision Sensor

At Hackaday we don’t often feature kickstarter campaigns, but this one is worth noticing in our opinion. It is called Pixy, a small camera board about half the size of a business card that can detect objects that you “train” it to detect.

Training is accomplished by holding the object in front of Pixy’s lens and pressing a button. Pixy then finds objects with similar color signatures using a dedicated dual-core processor that can process images at 50 frames per second. Pixy can report its findings, which include the sizes and locations of all detected objects, through one of several interfaces: UART serial, SPI, I2C, digital or analog I/O.

The platform is open hardware, its firmware is open source and GPL licensed, making the project very interesting. It is based on a 204MHz dual core ARM cortex M4 & M0, uses a 1280×800 image sensor and can stream the processed camera output to your computer. You can get one Pixy in the kickstarter campaign for $59, which is not that expensive for what it is.

45 thoughts on “A Fast And Easy-to-use Vision Sensor

    1. They reject the large majority of Kickstarter campaigns submitted to them, many of which are seeking to get free publicity. Only those of particular interest to the hacking community are accepted. I’d say this easily qualifies.

    1. Interesting. I got my vision processing start with CMUcam2 back in college. It was neat but limited. Now it seems hard to top an Android phone but I’ve always been frustrated by the inefficiency of that route. . . at mere $50 this is worth a try, in my not so humble opinion.

      What have you done with your CMUcam? Myself, not much.

      – Robot

  1. ..Also, one thing I don’t understand here is how it associates the color and shape to an object. So would this mean that if you have a cube with different colors per side, it would treat one side of that cube as a separate object?

    1. Sure. You could make a remote control by associating a channel to each colour, and flip the cube over to watch something else.

      Or a more useful function would be to recognise the difference between a full & empty glass, the ‘colour’ or the object will gradually change.

  2. This is the piece I need to finish my “Early Morning Cat Detection and Door Scratch Deterrence System.”

    My bloody cats all seem to think it’s okay to paw at the bedroom door when they get hungry, no matter what time it is. Little buggers.

    1. Never underestimate a water spray bottle filled with water and a little white vinager. They hate the smell.. My cats use to do this and other things like get on the bar, etc. Now when they see the “bottle” they run like H***

    2. Or leave the vacuum cleaner outside the door, turned on, with the cord run to a convenient switch by your bedside. Cats usually learn that attempting to wake you will result in waking the EEK (Evil Eater of Kitties) instead, and you don’t even have to get out of bed.

      1. Waking up everyone else in the house is not exactly the most effective way to deal with an untrained animal. There are any number of chemical deterrents that will prevent the cat from scratching the door without creating a public nuisance. Or you can feed your cat on a strict schedule and they will get used to that.

        1. Schedules haven’t worked. We’ve been feeding them at set times for several years. They persist in pawing/scratching at any door that keeps them from people.

          We’ve tried everything that’s carried at Petco and Pets-r-us.

          1. Have you tried the cardboard scratching pads? We sprinkle them with catnip and put them near the furniture, they go for the cardboard every time.

            What I have learned from cats is that negative feedback doesn’t work at all. They do not have human reasoning power and they do not “learn lessons” like humans do. If you want to get them to stop doing something then you must play to their immediate attention and give them something that they like better.

          2. Brother, they have all sorts of “approved” scratching spots and they make full use of them when there are people up and about. But as soon as they are alone and they know that there’s a door between them and their “property,” they don’t hesitate to register their discontent.

    3. I thought about how I might use this with my existing plans for a cat deterrent device, but I’m not sure it would be more useful than a simple PIR motion sensor. The PIR sensor works in total darkness and doesn’t care what color the object is.

      What are you planning to use as the actual deterrent? I have a simple circuit that uses a PIR sensor to trigger a relay attached to a blender that works pretty well for the kitchen, but obviously that’s not great for bedroom doors. My current plan (if I ever get around to finishing it) is to use a pair of extremely loud ultrasonic transducers that sweep up and down in frequency, a few hundred Hz apart so they produce a beat note as well. No idea how well it’ll work in practice, though.

      1. I’ve worked in the machine vision industry, and the #1 saying was “don’t use a machine vision system when a photocell will do the job just as well”

        Creative placement and tweaking of primitive optical and mechanical sensors can be remarkably inexpensive, effective, and reliable compared to the noisy, delayed data from software vision systems.

      2. I’ll be going the ultrasonic route as well.

        I’ve played the PIRs and wasn’t satisfied. They’re a PITA to properly shield to only protect the doorway and not a fair chunk of room.

    4. The cats could you know, just want either attention, or to cuddle. The other possibility is that they see the door closed and are confused why it was open but now isn’t. I keep my bathroom door open (I live alone) and when I have company (obviously closing it then) my cat will come scratch at the door.

      Leave the bedroom door open a crack. If you don’t like cats sleeping with you I don’t understand why you have one.

  3. This is the kind of kickstarter project that I admire; they’ve made and marketed previous design iterations, the current design is all done and debugged and manufacturable, they are just looking for front money so they can place big parts orders to get lower prices. None of the “oops we shipped untested product” mentality that I’ve been a victim of (you know who you are).

      1. This is true of every single computer I have ever purchased. My OSX laptop gets “little software clean-ups” all the time. If we waited for the software to be “finished” then we would never ever purchase computers.

        1. Hmmm… I seem to remember my Commodore 64 and 128 both being “done.” Same thing for my Atari ST.

          I realize that they might not have had as many features as a modern OS, but they also had fewer problems.

          1. Games used to be like this. The second highspeed internet became common place, day one patches and frequent bug and balance fixes became common place.

            Console games lasted longer until they got highspeed connectivity.

  4. I just built PIXY Cam and pan-tilt mechanism, Great hardware, goes together easily
    and has good instructions online. I use PixyMon under windows XP 64 to control
    the pixy.
    It’s tracking is wacky. Recognition is by color, and it is amazing how many of the
    same color objects are in the FOV, which sends it crazy trying to track.
    Recognition by color!!!
    My Pixycam pan-tilt only two days old
    Maybe it will get smarter?

Leave a Reply to professorKCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.