Thanks to modern microcontrollers, basic home automation tasks such as turning lights on and off, opening blinds, and various other simple tasks have become common DIY projects. But with the advent of artificial intelligence and machine learning the amount of tasks that can be offloaded to computers has skyrocketed. This shopping cart that automates away the checkout lines at grocery stores certainly fits into this category.
The project was inspired by the cashierless Amazon stores where customers simply walk into a store, grab what they want, and leave. This is made possible by the fact that computers monitor their purchases and charge them automatically, but creator [kutluhan_aktar] wanted to explore a way of doing this without a fleet of sensors and cameras all over a store. By mounting the hardware to a shopping cart instead, the sensors travel with the shopper and monitor what’s placed in the cart instead of what’s taken from a shelf. It’s built around the OpenMV Cam H7, a microcontroller paired with a camera specifically designed for these types of tasks, and the custom circuitry inside the case also includes WiFi connectivity to make sure the shopping cart can report its findings properly.
[kutluhan_aktar] also built the entire software stack from the ground up and trained the model on a set of common products as a proof-of-concept. The idea was to allow smaller stores to operate more efficiently without needing a full suite of Amazon hardware and software backing it up, and this prototype seems to work pretty well to that end. If you want to develop a machine vision project on your own with more common hardware, take a look at this project which uses the Raspberry Pi instead.
Aside from how carts are treated in general pretty much the same can be done with phone apps. The only bugaboo is things that need to be weighed.
So self scan each item as you place in cart, and the cart itself is a scale.
In Germany scales that are used for pricing have to be certified and calibrated regularly. So these carts would be quite expensive.
That’s right, but the idea here is to check if your chart is heavier than it should be (shoplifting), not to calculate the price.
Maybe in this case a simple scale could work.
I believe that’s the case in most places now, I know it is in the states, and many commercial scales I’ve seen explicitly state what functions are legal for trade. I would imagine that weighing everything at once would probably not be a legally excepted for determining price pretty much anywhere.
The simple solution is to not weigh anything. All vegetables have a unit/bundle cost, all meats are weighed ahead of time, etc. Bananas would be tricky but, I’m sure there is a solution, probably set bundle sizes and singles have a set price as well.
Most sundries all ready have a known weight for the self check out. The bag side has a scale. Tracking each package is best. Just have manufacturer make trackable bar codes. Home depot has unregulated self checkout with no weight and people take half the stuff. In America we have insurance.
Amazon’s version of this does have a certified and calibrated scale in the cart: https://www.aboutamazon.com/news/retail/amazon-dash-cart-new-features-whole-foods
Kroger has those carts. Bottom of cart is a weight sensor. Produce scales are connected devices so you weigh in the produce area and the item, price, weight gets added to your cart via a touchscreen on the cart.
Wegamans has self service scales in the produce dept. Type in the code on the sticker (4011), weight your bananas and stick the thermal paper sticker with the barcode right on the bunch (or produce bag)
When was the last time you bought something on Amazon by weight and not prepackaged?
From the article: “This shopping cart that automates away the checkout lines at grocery stores certainly fits into this category.”
From Amazon Fresh? All the time. And that’s what a trip to these grocery stores would generally be replacing: the fresh produce you can’t just have shipped to you by UPS.
Maybe in the future peolle who think they are shoplifting will find the amount deducted from their bank account.
San Fransisco can really use this technology right now.
We could use this for politicians as well.
LOL!
Kroger has these and is testing a LOT more automation in the Southwest Ohio region. We are less than 5 years out from employee-less grocery stores.
Smart shelves – weigh, scan, and track inventory levels.
Smart labels – small digital pricing screens, connected and able to be manually adjusted remotely.
KroGo carts – same as the ones Amazon used so you can scan, bag, and go.
Connected produce scales – scan on the certified scale and have it added to your KroGo shopping cart.
Drone delivery service (live in Centerville OH) with delivery based of phone GPS location so not just home delivery.
AGVs to unload, stock and pick orders.
No one will be needed store side – department managers, cashier’s, stockers, etc. Replace them with a few security, maintenance and IT folks servicing a few local stores and you’ve eliminated 95% of the workforce.
Quite a few of the major categories are Vendor Managed Inventories (VMI), so there’s minimal corporate infrastructure needed to maintain inventory. Large CPG and other supplies like Pepsi, Unilever, P&G, etc. would all happily own the VMI work to increase their respective sales.
really not sure if i like the idea of more surveillance and data collection in grocery stores
I eat seasonally and prepare my meals from scratch, so this would never work for me. Nearly all of my ingredients are priced by weight.
Secondarily I don’t like the direction that grocery stores are headed. The food you get pre-packaged is the nutritional equivalent of eating dog food, and usually comes from far away. This model of going cashierless only perpetuates the excessive substitution of whole foods for processed, shelf-stable counterparts. And the consumption of which leads to poor health across the world and a rise in MY insurance premiums.
Automation is neat stuff, but it should not be used to facilitate the sale of junk food. Focus on delivering better quality food to the consumer, not simply expediting transactions for things that fit well on a shelf.
Ironically, dog food is pretty heavily regulated and its nutritional content is good even on cheap brands. But yeah, I know what you mean.
Most supermarkets solve this by pre-packaging fresh ingredients into bags, eg 1kg bag of carrots. So this should work fine.
Oh, and supermarkets locally which have self-checkout trolleys (which they’ve had for years – barcode scanner in the trolley) just have weighing machines you use yourself and they spit out a barcode with item/weight in it.
Purina Bachelor Chow FTW!
Now with flavor!
“And the consumption of which leads to poor health across the world and a rise in MY insurance premiums.”
The health insurance I get through work charges smokers a much higher premium. Maybe the next step is to charge a higher premium for the obese, with an exemption for those unable to lose weight for medical reasons as well as those who demonstrate a trend of losing weight?
Oh great so a poor tax.
While cheaper foods can be more calorie dense, eating such a quantity of those foods to the point of obesity is a separate issue, unless you’re claiming all poor are somehow unable to understand portioning which I vehemently disagree with.
This is big box business model, if you want local fresh, support farmer’s markets
Honestly though, I don’t see why they can’t make a produce checkout robot..
Just went weekly grocery shopping at a mega mart. 75% of the basket was filled with not processed food. Just because it’s there doesn’t mean you have to buy it. But go enjoy your “whole paycheck” shopping
Ages ago, like in the early 2000’s, stop and shop had a thing where you pick up a price gun from the rack by the entrance and just scanned stuff as you put it in cart. Produce scales had a printer and would print a label that you scanned so weighing stuff wasn’t a problem. At the end you used the same scanner at a dedicated checkout line, scanned the barcode to say you were done then paid using whatever method you usually used. It was really awesome and was prior to dedicated self-checkout lines that are common now. I guess you could say it was a liability because you could just steal stuff but these days all stores have a policy that they can’t even try to stop people from stealing anyway.
Worked fine locally. It was tied to a loyalty scheme so they knew who you were, and they did spot checks the first few times and then every so often. Think it’s still running.
This is in basically every major supermarket in the UK and has been for at least a decade. Suspect it’s also true for most of the rest of Europe. A lot of people don’t use them but they are almost always there as an option.
It has been replaced with a phone app. Walmart and Sam’s Club use it. I’m a big fan. Nowadays, you scan everything, pay in the app, a single worker scans a QR code from your phone, they verify the total number of items in your cart and check some random items, and you’re out the door.
Is no one going to mention that the backplane PCB for this project is shaped like Scrooge McDuck? Or the gratuitous 3d printed box? What motivated these choices? These are more interesting to me than the rest of the project :)
Knowing how people treat trollies, it would be wildly more expensive than cameras around the store
Amazon already has shopping carts that track what people put into them and then allows them to skip the checkout line. For those who are interested in an in-person demo, the Amazon Fresh Store in Seattle on Jackson features this technology.
I’m a trifle confused. It uses a camera to detect goods, right? It looks at items when you put them in the trolley?
What happens if I change my mind, and take something out? And how does it compare to me adding TWO of something to the trolley? If I put something in a shopping bag, how does that impact the camera? What if the camera is obscured?
Well canadian supermarkets already have these carts where you pay at the cart
We need more innovations like this that reduce the number of minimum wage jobs in a society with a worker shortage this would make more good jobs for everyone therefor reducing poverty.