The Story Of Kickstarting The OpenMV

31 Comments

Robots are the ‘it’ thing right now, computer vision is a hot topic, and microcontrollers have never been faster. These facts lead inexorably to the OpenMV, an embedded computer vision module that bills itself as the ‘Arduino of Machine Vision.’

The original OpenMV was an entry for the first Hackaday Prize, and since then the project has had a lot of success. There are tons of followers, plenty of users, and the project even had a successful Kickstarter. That last bit of info is fairly contentious — while the Kickstarter did meet the minimum funding level, there were a lot of problems bringing this very cool product to market. Issues with suppliers and community management were the biggest problems, but the team behind OpenMV eventually pulled it off.

At the 2016 Hackaday SuperConference, Kwabena Agyeman, one of the project leads for the OpenMV, told the story about bringing the OpenMV to market:

The OpenMV is a computer vision module that’s very inexpensive, and also very simple. There are really only two main parts on this module: a powerful microcontroller, and a camera module able to capture images in native JPEG format. The reason the OpenMV is inexpensive is because the camera module itself is inexpensive — it’s a legacy image sensor that’s about a decade old, but if you’re doing computer vision on a microcontroller you really don’t need a lot of megapixels.

Those old camera sensors came back to bite the OpenMV project. Apparently, when you’re using 10-year-old BGA parts, sometimes the balls go bad. The OpenMV project saw 80% of their image sensors fail during assembly, and was even featured as a Hackaday Fail Of The Week.

Trying to get to the bottom of this problem, Kwabena called the manufacturer of these image sensors and discovered the problem. These image sensors were never sold on the open market, and only to OEMs. The most probable way these image sensors found their way into the OpenMV supply chain is that they were originally intended for phones, but they were either desoldered and placed into modules, or just kept as unused stock to be unloaded on AliExpress.

What was the solution to this problem? Unfortunately, another manufacturing run with a new camera module at the cost of about $18k. This was a success, and the OpenMV community got their new, upgraded computer vision module.

Now that OpenMV cameras are out in the wild and a community is growing up around them, the creators of this computer vision module decided to do a new iteration of their hardware. The new OpenMV is using a faster processor, has more RAM, and has a lower price than the original.

At Hackaday, we’re intimately familiar with what can go wrong in a manufacturing run. We’ve already seen dozens of Kickstarters fail because of bad luck or poor planning. The troubles OpenMV had couldn’t be planned, and it would take either luck or a lot of work to dig themselves out of the problems they had. They did it, and the result is a thriving community built around the hardware they created. That’s awesome, and one of the best hardware success stories we heard at the 2016 Hackaday SuperConference.

31 thoughts on “The Story Of Kickstarting The OpenMV”

  3. If you’re doing computer vision on a microcontroller, don’t expect it to be more than fairly basic at best. It’s better than not having it if you need it (sometimes) but it’s also not exactly able to physically crunch much either. Microcontrollers need to get a lot better at raw number crunching before any type of high (or even lowish) resolution image processing can be done. Let alone stereo image processing or any real “machine vision” tech. Better algorithms can help, but only to a point.

    Maybe once self driving car vision modules hit the secondary market, the prices for these kind of engineered systems will be reasonable enough to do something novel with them, sort of like how the multiple axis “motion” sensor hardware opened the door to all sorts of neat primary and secondary applications (Nintendo Wii, etc) but they had to actually be manufactured in bulk first to bring the sizes and costs down and make it so they were accessible and reliable.

    Report comment
    Reply

    1. You got to start somewhere. And now, even in a basic form, it has been shown to work, it can be improved on until it is at the level that it is useful for most applications. Saying no point using something until it’s got to the best possible iteration will mean it’ll never get to the best possible iteration.

      Report comment
      Reply

      1. Not trying to discourage early adoption by any means. Just sort of sitting here lamenting the fact that it’s almost 2017 and while we can get Arduinos and such for “fairly” cheap, it’s also 2017 and the Arduino UNO runs at 16MHz and has 32 KB of flash memory and 1 KB of EEPROM. These are not even MB figures. KB. Yes, it’s a microcontroller and not a general purpose device and yes, other platforms are better and yes, it can still do a lot but it just feels like we should be much, much further along than we are by now? Both from a hardware and software development perspective.

        Plus, to a degree, image processing isn’t going to *do* much until it has enough horsepower under the hood, so to speak. So iterating is great and absolutely helps but building the raw (and expensive and depreciating) hardware modules feels like what is really holding things back? Software to a degree as well.

        Report comment
        Reply

        1. Yeah but you can get ARMs for next to nothing, there’s that Raspberry Pi for $5 and a few others for less than $10. What do you want for your money!? Of course AVR chips aren’t going to get gigaflops any time soon, they’re designed for very low power. You’re looking in the wrong place!

          For a few quid more, Intel are doing entire PCs on a chip, and there’s mobile phone chips with 8 and 10 cores. For those, you might need to buy more than one or two.

          Report comment
          Reply

    2. Sadly no, the car vision systems are so freaking encoded and proprietary to keep other companies from buying a car and reverse engineering it is well beyond most hackers abilities right now. I was able to source some from a crashed audi and it’s lane assist system. everything potted, and the magic happens on FPGA’s and it’s loaded from the master processor box that I did not get.

      Report comment
      Reply

  4. I remember messing with the CMUcam 2 on Basic Stamp and running into the limitations pretty quick. I spent a lot of time wishing for something like this.

    Today, though, is there really any benefit to using OpenMV as opposed to a RasPi and USB webcam, both of which I can buy for well under OpenMV’s $50 price tag?

    Report comment
    Reply

      1. Hi Steven, this is Kwabena. The end goal is to make sub $25 camera modules (camera, lens, processor, and mezzanine connectors) which you can embed easily in products which are really low power. There’s a big need for super cheap, ease-to-use, small, and low-power camera modules to do rather boring applications. For example, let’s say I want to build a commercial product that needs to be able to read non-replaceable mechanical dials in a un-powered environment for years (i.e. water meter outside your house underground). Do I want to develop my application around a computer board that has an operating system, requires a lot of juice to run, and has in general way more power than I ever need? Or would I rather use a microcontroller that’s small, cheap, and that I have total control over?

        Another point would be usability… so, making an application with OpenCV and deploying it isn’t exactly a less than 100 LOC experience. One of the goals of OpenMV is to make it easy to program the thing. We’re trying to replicate the Arduino Microcontroller experience.

        Anyway, we have a good FAQ on the why of the system here: https://openmv.io/pages/faq

        Also, it’s not like we’re exactly on a low powered MCU anymore. The M7 is about equivalent to a Pentium 2 and the next version of the OpenMV Cam will use the STM32H7 processor which is as fast as a PI Zero. See here: http://www.st.com/en/microcontrollers/stm32h7-series.html?querycriteria=productId=SS1951.

        Report comment
        Reply

        1. Thanks for replying. Don’t take my criticism to heart, I wish you the best.

          I’m confused about your target market here, though.
          “The end goal is to make sub $25 camera modules (camera, lens, processor, and mezzanine connectors) which you can embed easily in products which are really low power.”
          Okay, so you’re going for a B2B model. It’s designed to be incorporated into other products produced at quantities where a $10/unit savings is meaningful, as opposed to hobbyists. That makes perfect sense.

          “We’re trying to replicate the Arduino Microcontroller experience.”
          Wait. So you ARE targeting hobbyists?

          Anyhow, I’ve been watching your progress on IO for a while. It’s very exciting to see a replacement for the CMUcam, even if I’m not sure where it fits in anymore.

          Report comment
          Reply

          1. Both actually. So, the highly integrated module I mention above would be available on attached dev board that looks like the current OpenMV Cam. Hobbyist/Students can use the system and if you want to make a product you buy the camera module yourself in bulk and then attach it to whatever.

            The primary goal of the whole project is to make computer vision easy. I see a lot of folks talking about how they can buy a cheap board that has a lot of power. But, I don’t see them talking about how many lines of code they have to write. Here’s an example of what we can do with the OpenMV Cam M7:

            QR Code Detection in 1 line: https://www.youtube.com/watch?v=mcIrKIMWhzs&t

            The project’s goal isn’t to… stay on a microcontroller. It’s to make computer vision easy that can control things in the real world. My partner, Ibrahim developed the system using the STM32 line of processors. So, that’s what we’re stuck on for now. But, maybe we will switch to the iMX processor line from NXP. At the end of the day the hardware is not important. It’s how easy we can make it for someone to get something working. If we had a full Linux computer w/ OpenCV we’d still try to hide all of that and wrap everything using MicroPython.

            Report comment

    2. Laaaaatencyyyy. The bottleneck are the 29fps and the lag of the USB bus. With the camera module things gets way better. Pi ~35€+20€ the camera module and you can cruch 60 fps motion detection(without streaming to screen). The cheap usb webcam usually have 200ms latency themselves.

      OpenMV is justified where low power and space is required.
      Also semplicity may be a premium. Rpi requires an hour or two to set up all the packets, while in that time you could have already started playing with your robot puppy backed by OpenMV.

      Report comment
      Reply

      3. You make great arguments, but I have to disagree on one point:
        “Rpi requires an hour or two to set up all the packets, while in that time you could have already started playing with your robot puppy backed by OpenMV.”

        To use a Pi, I just have to plug in a webcam, install OpenCV, and copy-paste some existing code and modify it to suit my needs. That’s not too hard since OpeCV is so widely used and there’s a ton of example code out there to adapt from.

        For OpenMV, I’m going to have to wait for it to actually get here (it’s crowdfunding so it might never actually materialize), write a script from scratch with MicroPython where I don’t have the benefit of popular Python modules, an OS, or even example code to get started with, and then I have to figure out how to get the OpenMV module to interface with my main logic board. Looks like it supports TTL serial, SPI, and I2C, so that’s good. Not sure how quickly I’d be able to pass video data over those channels, though, so my logic board is going to need its own camera anyway if I’m going to do anything like face recognition or machine learning. I assume OpenMV is more optimized for blob tracking, like the CMUcam was, as opposed to modern machine vision.

        Report comment
        Reply

  6. I would like to propose a moratorium on reporting on Kickstarter projects, since such a high percentage of them are either scams, incompetently run, or just stupid ideas. I backed about a dozen (mostly electronics projects) before calling it quits—which I guess makes ???????? stupid—and ended up regretting every one of them. Even the (few) good ideas are too often eclipsed by commercial products before they get to shipping, and the most enticing projects never ship at all (remember the Glowforge “3D laser printer”, for example? And the Shaper Origin? They’re still taking money!! I’ll bet Wazer is going to be in this category, too).

    Report comment
    Reply

  7. The openMV is absolutely amazing. Microcontrollers are getting more powerful….(just look at the stm32F7 or mother cortex-M4/M7 microcontrollers) and yes they can do at least lower end and intermediate computer vision tasks which is no small feat. Where microcontrollers lack is in a standardized operating system that comes with standard tools, software and libraries. And sure efforts such as mbed-os and zephyr are great but they’re still underdeveloped and not mature. The binary size of an mbed program that toggles some LEDs and sends/receives data over the UART can easily exceed 50KB.

    Microprocessor based systems (with MMUs) that can run Linux/BSD are much better suited for such tasks i.e. computer vision, networking. They have standardized software stacks that can be easily leveraged for such tasks, so no need to roll your own non-standardized stuff that will rarely/never get used again, they can come in very small packages almost as small as microcontroller based boards and they’re generally more powerful and flexible..and did i mention that they are very low cost nowadays? The developer can also use one of many programming languages / dev tools and is not limited to maybe one or two languages at most.

    In this day and age where you can get $9 CHIPs and $5 Raspberry Pi Zeros, I see microcontrollers primarily excelling in the hardware interfacing (digital I/O, analog I/O interfacing to other ICs) and hard real-time arenas.

    In my mind the ideal use case…let the Microprocessor running an OS do all the processor intensive stuff and stuff requiring use of fancy software stacks / libraries. It can then talk to a slave microcontroller that does all the hardware interfacing via some form of remote procedure calls.

    Report comment
    Reply

  8. I like the openMV, too.
    A global shutter sensor with higher frame rate would be nice for fast moving robots, though (e. g. OVM6211 or the MT9V034 of the PX4flow)
    And as for a lot of such products global availability is a problem. To get it in my hands I have probably to spend about 100 EUR, discuss a lot with customs and be patient.

    Report comment
    Reply

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.