Harvesting delicate fruit and vegetables with robots is hard, and increasingly us humans no longer want to do these jobs. The pressure to find engineering solutions is intense and more and more machines of different shapes and sizes have recently been emerging in an attempt to alleviate the problem. Additionally, each crop is often quite different from one another and so, for example, a strawberry picking machine can not be used for harvesting lettuce.
A team from Cambridge university, UK, recently published the details of their lettuce picking machine, written in a nice easy-to-read style and packed full of useful practical information. Well worth a read!
The machine uses YOLO3 detection and classification networks to get localisation coordinates of the crop and then check if it’s ready for harvest, or diseased. A standard UR10 robotic arm then positions the harvesting mechanism over the lettuce, getting force feedback through the arm joints to detect when it hits the ground. A pneumatically actuated cutting blade then attempts to cut the lettuce at exactly the right height below the lettuce head in order to satisfy the very exacting requirements of the supermarkets.
Rather strangely, the main control hardware is just a standard laptop which handles 2 consumer grade USB cameras with overall combined detection and classification speeds of about 0.212 seconds. The software is ROS (Robot Operating System) with custom nodes written in Python by members of the team.
Although the machine is slow and under-powered, we were very impressed with the fact that it seemed to work quite well. This particular project has been ongoing for several years now and the machine rebuilt 16 times! These types of machines are currently (2019) very much in their infancy and we can expect to see many more attempts at cracking these difficult engineering tasks in the next few years.
We’ve covered some solutions before, including: Weedinator, an autonomous farming ‘bot, MoAgriS, an indoor farming rig, a laser-firing fish-lice remover, an Aussie farming robot, and of course the latest and greatest from FarmBot.
Video after the break:
16 years to program a task an unskilled labourer can start doing in 15 minutes, the miracle of artificial intelligence.
Depending on the age of the labourer, they too have had at least 16 years worth of programming.
Teach a newborn to do it, and I will be impressed.
I just imagined a newborn harvesting lettuce like a robot. Thank you.
That is the beauty of it…. no matter how much effort is spent developing it, once it is done it can be multiplied at practically zero cost.
at hardware cost & maintenance
it similarly valid for HW as well. Everything off the shelf you buy shares a very small slice of the development cost required to design that product.
This can’t be right -they’re in England they aren’t wearing white lab coats
Dan and Gav only tend to wear those in the USA now.
The hi-viz vest is far more common among British scientists of all genres. Elthandsafetymate.
I don’t think its a case of people don’t want to do the work, it just pays very poorly and is seasonal so what do you do the rest of the time.
Work in that season´s crops.
The basic idea shouldn´t be for it to be considered as a profession, just as a “make ends meet” work. Same as flipping burgers at McDonalds. The kind of work people in the down can do and earn a little, or that people without much knowledge, or language barrier, can simply do. Better than just having to receive government money.
It’s much more complicated than that. Sure, flipping burgers is not that hard, but for any job you need a bunch of things in place to be reliable and effective. Consider transportation, a place to sleep, adequate food so you have the energy to work, adequate healthcare so that you’re physically able to do the job. If you’re starting at 0 then a lot of those things are out of reach. That’s why I support a large increase in the minimum wage. I’d rather pay more at the checkout by having to pay a little more than have it funneled through government programs and pay those costs through my tax dollars.
When it returns to base with a cargo of lettuce-sized human heads don’t say I didn’t warn you.
“and increasingly us humans no longer want to do these jobs. ”
I’ll be changing that phrase with something more real:
… and increasingly greedy humans no longer want to PAY to have these jobs done.
Plenty of humans out there will want to do this as a paid job.
Lets call a spade a spade, please.
“Who will pick the cotton when we have no slaves?” Funny how the the cheap lettuce crowd still thinks this way.
Think it used to be kids would do this kind of work for cash. Now workers have to be adults, minimum wages, and taxed.
Right on the nose. Not so much people don’t want to do the jobs; someone just wants more profits and thinks they can come out ahead investing in robotics.
I remember reading an article in the Wall Street Journal about 10 years ago: A watermelon farmer decided to build a watermelon-harvesting robot after losing his entire crop one year because he couldn’t hire workers to do do the harvesting.
After investing over a million dollars in his robot system, it was not even half as efficient as a human worker, yet he was committed to continuing his development because he never wanted to lose a crop that way again.
The problems were exactly as described in this article: Human workers could be trained in a short time and distinguish from sight sound and feel whether the watermelon was ripe for harvesting or not. Human workers could handle the crop in ways that caused less damage.
This one article caused me to look at agri-robotics in a completely different light. (I had never actually considered the scope of agri-robotics applications before.) How much does it cost to develop a task-specific robot, and then how wide-spread must it be used, and at what price, before it is cost effective? How reliable must it be? Most importantly (to me) what kind of thinking must we teach people to design machinery to overcome the obstacles? What about multi-tasked robots? For instance, what kind of machine would be a robotic cowboy, capable of herding, branding, castrating, dipping, and all the other tasks involved with cattle? How long before there is even a machine to shear sheep?
There is a big gap between the state-of- the-art robotics industry and the ordinary low-tech abilities of human ingenuity.
“There is a big gap between the state-of- the-art robotics industry and the ordinary low-tech abilities of human ingenuity.”
Maybe 10 years ago, now? Much less. You can now build a soft picker using computer vision and deep learning quite easily. It is still not to the level of an trained adult, but I’m pretty sure that one could devise a deep learning program that could analyse melon ripeness level and train it to become efficient at recognizing ripe fruit.
It’s a matter of time until robo-farming become mainstream and overtake most of farm jobs.
The time is ripe for agri-bots!!!!
And….
That reminds me of the opening scene of the movie: “Runaway” with Tom Selleck
Look for ABB Flexpicker on YouTube. You’ll see robots rapidly handling odd shaped items and soft food products. Now think of one of those mounted in a way that it can be pivoted on the upper end end of an arm so that it can cover nearly a full sphere.
Mount it on a wheeled base to run down the rows of a fruit orchard. With cameras using colorimeter tech to measure size and ripeness by color, it could rapidly triage a citrus crop to pick over ripe, ripe, and ripe enough for shipment fruit, while leaving the not ripe enough fruit on the trees. There’s likely some data to be tested by shining a LASER across the peel to see how much light goes through as a test for ripeness.
Many orchards use elevated sickle bar mowers to top their trees so pickers don’t have to climb ladders over a certain height. Robotic picking could be aided by also cutting the sides of the trees to make them have a cube shape. Then run the robots both ways across the orchard. Could mount several on gantries so they could pick the tops and drop down between the trees to pick the sides.
in tokyo since many years
A machine like this should be adaptable to harvesting iceberg lettuce. It uses water jets to slice green leafe anf romaine lettuce free of its root http://www.ramsayhighlander.com/products/romaine/green-leaf-lettuce-harvester.htm
That’s a $500,000 ish monster!