OpenCV And Depth Camera Spots Weeds

Using vision technology to identify weeds in agriculture is an area of active development, and a team of researchers recently shared their method of using a combination of machine vision plus depth information to identify and map weeds with the help of OpenCV, the open-source computer vision library. Agriculture is how people get fed, and improving weed management is one of its most important challenges.

Many current efforts at weed detection and classification use fancy (and expensive) multispectral cameras, but PhenoCV-WeedCam relies primarily on an OAK-D stereo depth camera. The system is still being developed, but is somewhat further along than a proof of concept. The portable setups use a Raspberry Pi, stereo camera unit, power banks, an Android tablet for interfacing, and currently require an obedient human to move and point them.

It’s an interesting peek at the kind of hands-on work that goes into data gathering for development. Armed with loads of field data from many different environments, the system can use the data to identify grasses, broad leaf plants, and soil in every image. This alone is useful, but depth information also allows the system to estimate overall plant density as well as try to determine the growth center of any particular plant. Knowing that a weed is present is one thing, but to eliminate it with precision — for example with a laser or mini weed whacker on a robot arm — knowing where the weed is actually growing from is an important detail.

PhenoCV-WeedCam (GitHub repository) is not yet capable of real-time analysis, but the results are promising and that’s the next step. The system currently must be carried by people, but could ultimately be attached to a robotic platform made specifically to traverse fields.

25 thoughts on “OpenCV And Depth Camera Spots Weeds

  1. Humans do the same thing with three channels of color and no depth sensor, so it’s likely this is a dead end and will be replaced with a pure rgb deep learned computer vision system if there is any money to be made.

      1. Stereoscopic vision is overrated, most people don’t use it much, as you can easily prove to yourself by closing one eye and walking around the house. At the distance for normal weeding it’s going to be almost entirely color and shape.

        1. Humans without stereoscopic vision can compensate with additional information such as knowledge of movement.
          Having worked with a guy with only one effective eye, it was interesting observing his head movement when he was estimating distances.

          1. Chicken vision, ever notice how they move their heads back and forward? They are integrating views in the same way to build a 3D model in their mind.

        2. The neuropsychology of stereoscopic vision is actually and area that I spent a fair chunk of 2020 working on. Found a couple of neat brain hacks too including inducing dynamic 3D perceptions from static 2D images. What we did find is that there is a lot of variability in terms of how good people’s functionality is in that area. However all that aside, isn’t your argument the same as saying two legs are overrated as most people can hop on one leg?

          1. I find, in random experimentation with me; that “considered” actions are pretty easy with one eye closed or covered, but rapid “instinctual” actions like catching something are more prone to failure or difficulty. However, I don’t know if they would improve with more practice. I also don’t know if an eye injury as a kid, necessitating having an eye covered for a week, made me a little better at one eyed navigation than typical, due to occurring when my brain was more neuroplastic as it were.

  2. When I was a kid, my siblings and I would walk the corn and bean fields of our parents’ farm, pulling weeds. At the time, WE were the weed identifiers. It was not the most pleasant task, generally carried out for hours at a time under the hot sun in the middle of the growing season, but it was more organic (and cheaper) than weedkiller, and it instilled a sense of pride and responsibility, contributing to operating a business.

    These days, I pull weeds from my backyard bushes and groundcover beds. Often the “weeds” are individual grass plants poking out from a compact shrub. (My Dad often told us that “a weed is a plant out of place.”)

    Trying to pull grass plants by grabbing the exposed grass leaves and pulling is doomed to failure. Inevitably, the grass plant will break at some middle point, and the removed part will just grow back. One really DOES need to trace the grass stem to its base and pull from there to extract the roots. I hope that the camera system designers will remember this.

    In any event, I wish them luck. Introducing an inexpensive and effective automated weed puller will cut down on child labor. It’s a nice memory, nonetheless.

      1. Roundup is forbidden for a reason in many country, nobody should use that nasty thing (and yes without it we wouldn’t be able to do large scale argiculture and nourrish all humans but the problem is too many humans, not the other way around).

        > a weed is a plant out of place
        I get what you father was saying but technically if the “weed” is growing somewhere it’s serving a purpose; often they filter soil from nasty chemicals and such so they have a purpose (even if they can look disgraceful)

        1. Because they’re spraying all over everything, crops (which then people eat) included. Glykofosphates (like Roundup…) need to be applied to any green part of the plant to kill it. Keep it off the crops = no immediate danger. Use as little as possible (targeted application) = little to no long term danger.

        2. We wouldn’t be able to do large scale agriculture *with as little manpower*. Stateside, regulations have been squeezing out small farmers for generations, and politicians have encouraged programs to keep food cheap for just as long. Hungry voters don’t tend to vote for the incumbents. It’s led to bigger farms, thinner margins (as the cost of inputs aren’t artificially kept low, but outputs are), more mechanization, and more pesticide and herbicide use. The latter two trends reduce human input. Human input is expensive (so long as it’s legal citizens, don’t get me started on what migrant workers lives look like).

          There’s a huge backstory that makes this kind of development a little disturbing to me, but also a little hopeful as it could reduce herbicide inputs to farming. Automation will either enable the ultra rich to fully disregard the poor, or get us to a Trek like utopia where people work on what they want because basic needs have been met.

          1. It would be nice to see this targeted against the invasive plants. Eliminating those sees less effort and there is _no_ money to be made eradicating these. Food plants are more complicated from many perspectives – as you accurately point out. Heading out to pull garlic mustard now.

        1. I was thinking that too, the current is sure to damage the roots. The trick is to get the nature of the pulse well matched to the conductivity of the target weed. Perhaps weeding when the soil is drier would be wise, then the current flow is going to spend more time inside the plant that travel deeper into the root system.

        2. I wonder about fire risk. A green plant is not going to be an issue, but high voltage around mature wheat plants seems risky. Which is too bad, because tasing weeds out of existence is the coolest option.

    1. Grass grows from the base, much like hair, hence the observed futility.

      The problem with using children as weed removal agents is that it doesn’t scale and the maintenance costs are pretty high.

Leave a Reply to PoppieCancel reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.