robot vision

3 Articles

Robots Learning To Understand Their Surroundings

November 6, 2020 by 1 Comment

Today it is pretty easy to build a robot with an onboard camera and have fun manually driving through that first-person view. But builders with dreams of autonomy quickly learn there is a lot of work between camera installation and autonomously executing a “go to chair” command. Fortunately we can draw upon work such as View Parsing Network by [Bowen Pan, Jiankai Sun, et al]

When a camera image comes into a computer, it is merely a large array of numbers representing red, green, and blue color values and our robot has no idea what that image represents. Over the past years, computer vision researchers have found pretty good solutions for problems of image classification (“is there a chair?”) and segmentation (“which pixels correspond to the chair?”) While useful for building an online image search engine, this is not quite enough for robot navigation.

A robot needs to translate those pixel coordinates into real-world layout, and this is the problem View Parsing Network offers to solve. Detailed in Cross-view Semantic Segmentation for Sensing Surroundings (DOI 10.1109/LRA.2020.3004325) the system takes in multiple camera views looking all around the robot. Results of image segmentation are then synthesized into a 2D top-down segmented map of the robot’s surroundings. (“Where is the chair located?”)

The authors documented how to train a view parsing network in a virtual environment, and described the procedure to transfer a trained network to run on a physical robot. Today this process demands a significantly higher skill level than “download Arduino sketch” but we hope such modules will become more plug-and-play in the future for better and smarter robots.

[IROS 2020 Presentation video (duration 10:51) requires free registration, available until at least Nov. 25th 2020. One-minute summary embedded below.]

Continue reading “Robots Learning To Understand Their Surroundings” →

An Optical Mouse Sensor For Robotic Vision

July 6, 2018 by 22 Comments

Readers with long memories will remember the days when mice and other similar pointing devices relied upon a hard rubber ball in contact with your desk or other surface, that transmitted any motion to a pair of toothed-wheel rotation sensors. Since the later half of the 1990s though, your rodent has been ever significantly more likely to rely upon an optical sensor taking the form of a small CCD camera connected to motion sensing electronics. These cameras are intriguing components with applications outside pointing devices, as is shown by [FoxIS] who has used one for robot vision.

The robot in question is a skid-steer 4-wheeled toy, to which he has added an ADNS3080 mouse sensor fitted with a lens, an H-bridge motor driver board, and a Wemos D1 Mini single board computer. The D1 serves a web page showing both the image from the ADNS3080 and an interface that allows the robot to be directed over a network connection. A pair of LiPo batteries complete the picture, with voltage monitoring via one of the Wemos analogue pins.

The ADNS3080 is an interesting component and we’d love see more of it. This laser distance sensor or perhaps this car movement tracker should give you some more info. We’ve heard rumors of them being useful for drones. Anyone?

Skittles Sorter

Only Eat Red Skittles? We’ve Got You Covered.

September 8, 2015 by 19 Comments

Are you a bit obsessive compulsive with lots of certain things? We are too. Like Skittles! If you’re the kind of person who likes to sort their Skittles, you should seriously look into making your own 3D printed Skittles Sorter.

Built more to challenge his new 3D printer, [MrPrezident] was looking for a project to combine mechanical design with a bit of image recognition prowess — so he came up with this clever, and compact, Skittle sorting machine.

It uses an Arduino Uno with a ZITRADES color sensor module to identify the color of each candy. A small LED helps illuminate the Skittles to ensure an accurate color reading. Then, depending on the color, a series of gears rotate the Skittles piece to its designated color repository.

Theoretically it should also work with M&M’s (which are a bit smaller) but unfortunately, there are 6 colors of M&M’s and only 5 colors of Skittles. What would the machine do then!? We don’t see a reject bin!

Continue reading “Only Eat Red Skittles? We’ve Got You Covered.” →