Increasingly these days drones are being used for urban surveillance, delivery, and examining architectural structures. To do this autonomously often involves using “map-localize-plan” techniques wherein first, the location is determined on a map using GPS, and then based on that, control commands are produced.
A neural network that does steering and collision prediction can compliment the map-localize-plan techniques. However, the neural network needs to be trained using video taken from actual flying drones. But generating that training video involves many hours of flying drones at street level putting vehicles and pedestrians at risk. To train their DroNet, Researchers from the University of Zurich and the Universidad Politecnica de Madrid have come up with safer sources for that video, video recorded from driving cars and bicycles.
For the drone steering predictions, they used over 70,000 images and corresponding steering angles from the publically available car driving data from Udacity’s Open Source Self-Driving project. For the collision predictions, they mounted a GoPro camera to the handlebars of a bicycle and drove around a city. Video recording began when the bicycle was distant from an object and stopped when very close to the object. In total, they collected 32,000 images.
To use the trained network, images from the drone’s forward-facing camera were fed into the network and the output was a steering angle and a probability of collision, which was turned into a velocity. The drone remained at a constant height above ground, though it did work well from 1.5 meters to 5 meters up. It successfully navigated road lanes and avoided moving pedestrians and bicycles. Intersections did confuse it though, likely due to the open spaces messing with the collision predictions. But we think that shouldn’t be a problem when paired with map-localize-plan techniques as a direction to move through the intersection would be chosen for it using the location on the map.
As you can see in the video below, it not only does a decent job of flying down lanes but it also flies well in a parking garage and a hallway, even though it wasn’t trained for either of these.
The IMAV (International Micro Air Vehicle) conference and competition is a yearly flying robotics competition hosted by a different University every year. AKAMAV – a university student group at TU Braunschweig in Germany – have written up a fascinating and detailed account of what it was like to compete (and take first place) in 2016’s eleven-mission event hosted by the Beijing Institute of Technology.
AKAMAV’s debrief of IMAV 2016 is well-written and insightful. It covers not only the five outdoor and six indoor missions, but also details what it was like to prepare for and compete in such an intensive event. In their words, “If you share even a remote interest in flying robots and don’t mind the occasional spectacular crash, this place was Disney Land on steroids.”
20 students of the Eindhoven University of Technology (TU/e) in the Netherlands share one vision of the future: the fully domesticated drone pet – a flying friend that helps you whenever you need it and in general, is very, very cute. Their drone “Blue Jay” is packed with sensors, has a strong claw for grabbing and carrying cargo, navigates autonomously indoors, and interacts with humans at eye level.
[Geir] has created a pretty neat device, it’s actually his second version of an autonomous boat that maps the depths of lakes and ponds. He calls it the Sea Rendering. The project is pretty serious as the hull was specially made of fiberglass. The propulsion is a simple DC motor and the rudder is powered by an RC servo. A light and flag adorn the top deck making the small craft visible to other larger boats that may be passing by. Seven batteries are responsible for all of the power requirements.
The craft’s course is pre-programmed in Mission Planner and uses ArduPilot loaded on an Arduino to steer to the defined way points. An onboard GPS module determines the position of the boat while a transducer measures the depth of the water. Both position and depth values are then saved to an SD card. Those values can later be imported into a software called Dr Depth that generates a topographic map of the water-covered floor.
[Geir] has sent this bad boy out on an 18 km journey passing through 337 way points. That’s pretty impressive! He estimates that the expected run time is 24 hours at a top speed of 3 km/h, meaning it could potentially travel 72 km on a single charge while taking 700 depth measurements during the voyage.
The SkyProwler uses a switch blade type mechanism to move from quadcopter mode to plane mode. The wings can be detached to make it a normal quad that has all the typical bells and whistles. It can follow you around with GPS, fly autonomously via way points, and has this cool gimbal mechanism that keeps the GoPro stable as the drone pitches in flight, allowing for a better video experience.
[Dan’s] ultimate goal is a full size passenger model called the SkyCruiser, which uses the same switchblade transformation mechanism as his much smaller SkyProwler. Be sure to check out the video below if you haven’t already, and let us know of any quadcopter / plane hybrids of your own.
Correction: We previously associated [Daniel Lubrich] with the ATMOS program. This was in error and has been removed from the article. The ATMOS UAV is a separate project which we previously covered.
Controlling autonomous vehicles remotely with the use of virtual reality headsets seems like an obvious next step. Already, a few UAV companies have begun experimenting with these types of ideas by integrating Oculus Rift developer kits into their hovering quadcopters and drones. Parrot released a video on their blog showing that they developed a head-tracking system for their Bebop Drone in an effort to bring FPV flights to fruition. It looks like a lot of fun and we want to try one of these out asap!
As for technical specifications, they can be found in the YouTube description of the video embedded below. A quick glance showed that the operating system is based on Android and uses WiFi to connect the handheld tablet to the autonomous vehicle floating above. The range is a whopping 2km, giving plenty of freedom to explore. Moving one’s head swivels the attached camera giving a more immersive flying experience.
This isn’t the first example of FPV drones that we have seen. Previously, we covered an Oculus Rift + Head Tracking setup and another similar integration with a Black Armor Drone. We are bound to see virtual reality equipment used to control drones more and more as developers get their hands on cutting edge hardware like the Oculus developer kit 2 hardware which is currently shipping.