Robot Allows Remote Colleagues To Enjoy Office Shenanigans

[Esther Rietmann] and colleagues built a Telepresence Robot to allow work at home teammates to have a virtual, but physical presence in the office. A telepresence robot is like a tablet mounted on a Roomba, providing motion capability in addition to an audio/video connection. Built during a 48 hour hackathon, it is a bit crude under the hood and misses out on some features, such as a bidirectional video feed. But overall, it pretty much does what is expected from such a device.

The main structure is build from cheap aluminium profiles and sheets. A Raspberry Pi is at the heart of the electronics hardware, with a servo mounted Pi-camera and speaker-microphone pair taking care of video and audio. The two DC motors are driven by H-bridges controlled from the Pi and an idle swivel caster is attached as the third wheel. The whole thing is powered by a power bank. The one important thing missing is an HDMI display which can show a video feed from the remote laptop camera. That may have been due to time constraints, but this feature should not be too difficult to add as a future upgrade. It’s important for both sides to be able to see each other.

The software is built around WebRTC protocol, with the WebRTC Extension from UV4L doing most of the heavy lifting. The UV4L Streaming Server not only provides its own built-in set of web applications and services, but also embeds a general-purpose web server on another port, allowing the user to run and deploy their own custom web apps. This allowed [Esther Rietmann]’s team to build a basic but functional front-end to transmit data from the remote interface for controlling the robot. The remote computer runs a Python control script, running as a system service, to control the drive motors and camera servo.

The team also played with adding basic object, gesture and action recognition features. This was done using PoseNet – a machine learning model, which allows for real-time human pose estimation in the browser using TensorFlowJS – allowing them to demonstrate some pose detection capability. This could be useful as a “follow me” feature for the robot.

Another missing feature, which most other commercial telepresence robots have, is a sensor suite for collusion avoidance, object detection and awareness such as micro switches, IR / ultrasonic detectors, time of flight cameras or LiDAR’s. It would be relatively easy to add one or several sensors to the robot.

If you’d like to build one for yourself, check out their code repository on Github and the videos below.

Continue reading “Robot Allows Remote Colleagues To Enjoy Office Shenanigans”

A Car Phone — No, Not That Kind

Autonomous vehicle development is a field of technology that remains relatively elusive to the average hacker, what with the needing a whole car and all. Instead of having to deal with such a large scale challenge, [Piotr Sokólski] has instead turned to implementing the same principles on the scale of a small radio-controlled car.

Wanting to lower the barrier of entry for developing software for self-driving cars, he based his design off of something you’re likely to have lying around already: a smartphone. He cites the Google Cardboard project for his inspiration, with how it made VR more accessible without needing expensive hardware. The phone is able to control the actuators and wheel motors through a custom board, which it talks to via a Bluetooth connection. And since the camera points up in the way the phone is mounted in the frame, [Piotr] came up with a really clever solution of using a mirror as a periscope so the car can see in front of itself.

The software here has two parts, though the phone app one does little more than just serve as an interface by sending off a video feed to be processed. The whole computer vision processing is done on the desktop part, and it allows [Piotr] to do some fun things like using reinforcement learning to keep the car driving as long as possible without crashing. This is achieved by making the algorithm observe the images coming from the phone and giving it negative reward whenever an accelerometer detects a collision. Another experiment he’s done is use a QR tag on top of the car, visible to a fixed overhead camera, to determine the car’s position in the room.

This might not be the first time someone’s made a scaled down model of a self-driving vehicle, though it’s one of the most cleverly-designed ones, and it’s certainly much simpler than trying to do it on a full-sized car in your garage.

Continue reading “A Car Phone — No, Not That Kind”

Home Automation At A Glance Using AI Glasses

There was a time when you had to get up from the couch to change the channel on your TV. But then came the remote control, which saved us from having to move our legs. Later still we got electronic assistants from the likes of Amazon and Google which allowed us to command our home electronics with nothing more than our voice, so now we don’t even have to pick up the remote. Ushering in the next era of consumer gelification, [Nick Bild] has created ShAIdes: a pair of AI-enabled glasses that allow you to control devices by looking at them.

Of course on a more serious note, vision-based home automation could be a hugely beneficial assistive technology for those with limited mobility. By simply looking at the device you want to control and waving in its direction, the system knows which appliance to activate. In the video after the break, you can see [Nick] control lamps and his speakers with such ease that it almost looks like magic; a defining trait of any sufficiently advanced technology.

So how does it work? A Raspberry Pi camera module mounted to a pair of sunglasses captures video which is sent down to a NVIDIA Jetson Nano. Here, two separate image classification Convolutional Neural Network (CNN) models are being used to identify objects which can be controlled in the background, and hand gestures in the foreground. When there’s a match for both, the system can fire off the appropriate signal to turn the device on or off. Between the Nano, the camera, and the battery pack to make it all mobile, [Nick] says the hardware cost about $150 to put together.

But really, the hardware is only one small piece of the puzzle in a project like this. Which is why we’re happy to see [Nick] go into such detail about how the software functions, and crucially, how he trained the system. Just the gesture recognition subroutine alone went through nearly 20K images so it could reliably detect an arm extended into the frame.

If controlling your home with a glance and wave isn’t quite mystical enough, you could always add an infrared wand to the mix for that authentic Harry Potter experience.

Continue reading “Home Automation At A Glance Using AI Glasses”

Build Your Own Selfie Drone With Computer Vision

In late 2013 and early 2014, in the heady days of the drone revolution, there was one killer app — the selfie drone. Selfie sticks themselves had already become a joke, but a selfie drone injected a breath of fresh air into the world of tech. Fidget spinners had yet to be invented, so this is really all we had. It wasn’t quite time for the age of the selfie drone, though, and the Lily camera drone — in spite of $40 Million in preorders — became the subject of lawsuits, and not fines from the FAA.

Technology marches ever forward, and now you can build your own selfie drone. That’s exactly what [geaxgx] did, although this build uses a an off-the-shelf drone with custom software instead of building everything from scratch.

For hardware, this is a Ryze Tello, a small, $100 quadcopter with a front-facing camera. With the right libraries, you can stream images to a computer and send flight commands back to the drone. Yes, all the processing for the selfie drone happens on a non-flying computer, because computer vision takes processing power and battery life.

The software comes from CMU’s OpenPose library, a real-time solution for detecting a body, face, or hands. With this, [geaxgx] was able to hover the drone and keep his face in the middle of the camera’s frame. While there’s no movement of the drone involved — the drone is just hovering and rotating to the left and right — it is a flying selfie stick without the stick. You can check out the video below and check out all the code on [geaxgx]’s GitHub here.

Continue reading “Build Your Own Selfie Drone With Computer Vision”

Finding Plastic Spaghetti With Machine Learning

Among 3D printer owners, “spaghetti” is the common term for the tangled mess of stringy plastic that’s often the result of a failed print. Fear of their print bed turning into a hot plate of PLA spaghetti is enough to keep many users from leaving their machines operating overnight or while they’re out of the house. Accordingly, we’ve seen a number of methods that allow the human operator to watch their print remotely to make sure everything is progressing smoothly.

But unless you plan on keeping your eyes on your phone the entire time you’re out of the house, there’s still a chance some PETG pasta might sneak its way out. Enter the Spaghetti Detective, an open source project that lets machine learning take over when you can’t sit watching the printer all day. Their system plugs into Octoprint to monitor your print in real-time and pause it if it starts looking particularly stringy. The concept is still under development, but judging by the gallery of results submitted by users, the system seems to have a knack for identifying non-edible noodles.

Once the software comes out of beta it looks like the team is going to try to monetize it by providing hosting and monitoring services for a monthly fee, but as it’s an open source project, you’re also able to run the software on your own machine. Though the documentation notes that the lowly Raspberry Pi doesn’t have quite what it takes to handle the image recognition routines, so you’ll need a proper computer if you want to self-host the service. Could be a good use for that old laptop you’ve got kicking around the lab.

As demonstrated in the video after the break, the system’s “spaghetti confidence” is shown with a simple to understand gauge: green is a good-looking print, and red means the detective is getting a sniff of the stringy stuff. If your print dips into the red too much, Octoprint is commanded to pause the print. The user can then look at the last image from the printer and decide to either cancel the print entirely, or resume if the Spaghetti Detective got a little overzealous.

Frankly, it’s a brilliant idea and we’re very interested to see where it goes from here. Assuming you’ve got Octoprint controlling your 3D printer there are some very clever monitoring systems out there currently, but since spaghetti isn’t the only thing a rogue 3D printer can cook up, having an extra line of defense sounds like a good idea to us.

Continue reading “Finding Plastic Spaghetti With Machine Learning”

Pixy2 Is Super Vision For Arduino Or Raspberry Pi

A Raspberry Pi with a camera is nothing new. But the Pixy2 camera can interface with a variety of microcontrollers and has enough smarts to detect objects, follow lines, or even read barcodes without help from the host computer. [DroneBot Workshop] has a review of the device and he’s very enthused about the camera. You can see the video below.

When you watch the video, you might wonder how much this camera will cost. Turns out it is about $60 which isn’t cheap but for the capabilities it offers it isn’t that much, either. The camera can detect lines, intersections, and barcodes plus any objects you want to train it to recognize. The camera also sports its own light source and dual servo motor drive meant for a pan and tilt mounting arrangement.

Continue reading “Pixy2 Is Super Vision For Arduino Or Raspberry Pi”

The Tiniest Computer Vision Platform Just Got Better

The future, if you believe the ad copy, is a world filled with cameras backed by intelligence, neural nets, and computer vision. Despite the hype, this may actually turn out to be true: drones are getting intelligent cameras, self-driving cars are loaded with them, and in any event it makes a great toy.

That’s what makes this Kickstarter so exciting. It’s a camera module, yes, but there are also some smarts behind it. The OpenMV is a MicroPython-powered machine vision camera that gives your project the power of computer vision without the need to haul a laptop or GPU along for the ride.

The OpenMV actually got its start as a Hackaday Prize entry focused on one simple idea. There are cheap camera modules everywhere, so why not attach a processor to that camera that allows for on-board image processing? The first version of the OpenMV could do face detection at 25 fps, color detection at more than 30 fps, and became the basis for hundreds of different robots loaded up with computer vision.

This crowdfunding campaign is financing the latest version of the OpenMV camera, and there are a lot of changes. The camera module is now removable, meaning the OpenMV now supports global shutter and thermal vision in addition to the usual color/rolling shutter sensor. Since this camera has a faster microcontroller, this latest version can support multi-blob color tracking at 80 fps. With the addition of a FLIR Lepton sensor, this camera does thermal sensing, and thanks to a new library, the OpenMV also does number detection with the help of neural networks.

We’ve seen a lot of builds using the OpenMV camera, and it’s getting ot the point where you can’t compete in an autonomous car race without this hardware. This new version has all the bells and whistles, making it one of the best ways we’ve seen to add computer vision to any hardware project.