Augmented reality UAV controller

Controlling a long-range unmanned aerial vehicle is much easier if you have an augmented reality system like [Fabien Blanc-Paques] built. On board the aircraft you’ll find a sensor suite and camera, both transmitting data back to the operator. As the title of this post indicates, the display the operator sees is augmented with this data, including altitude, speed, and a variety of super-handy information. For instance, if you get disoriented during a flight there’s an arrow that points back to home. There’s also critical information like how many milliamp-hours have been used so that you can avoid running out of juice, and GPS data that can be used to locate a downed aircraft. Check out some flight video after the break.

45 thoughts on “Augmented reality UAV controller

  1. That setup turns rc planes from something I find mildly interesting to something I find totally badass. It’s like you’re actually flying! If you had the money to burn you could equip a couple of them with BB-gun cannons for aerial dogfights.

  2. this looks awesome. question though. is there a particular reason he opted for a camera above as opposed to below the plane? is it a functional decision or is it of no consequence?

  3. Interesting hardware but as already said, this is not AR, simply an on-screen display. AR systems add digital objects to the user’s field of view. This system only displays textual information, and even then, over the video feed from a remote device rather than the first person perspective of the user.

  4. @phil
    Beyond being a more pilot-like position to view the plane from, putting the camera on top means you can belly-land the plane without risking damage to the camera. These types of planes generally don’t have landing gear, and judging by the way it was sitting at the start of the video, this one is no exception.

  5. Even if someone publishes a hack showing how to connect to the cerebral cortex and lets him…

    Well, in my book displaying blocky text most certainly counts as A/R. No, it’s not as cool as using a polhemus unit to control the plane instead of an R/C transmitter, but this is well done and certainly augments reality. His brain was looking out that plane’s virtual cockpit with a HUD – from the ground.

    Could it be cooler, use a power glove and burn a ton of graphics power to make something that looks like a circa 1997 virtual reality project? Sure. But why bother?

  6. PS – @MSGFX I’m a talkin’ to you:

    >AR systems add digital objects to the user’s field of view.

    Your view of AR is too narrow. I understand that you want this system to include video-game like graphics and Synthetic Vision features before you’ll call it AR, but I beg to differ.

  7. Please, this is not AR!

    If so, any on screen display information would AR, which is not!
    AR has to include real world information with virtual environment information… so where is the virtual???
    GPS… No!
    Distance and orientation to home… “Probably” can be AR.
    Battery power, clock, date… No!

    This is simply: RC with FPV/RPV (first person view/remote pilot view) and with OSD (on screen display).

  8. To me, the yellow first down line on NFL broadcasts is AR the score board at the top of the screen is not.

    The arrow home is close but really it would need to have the actually image of the runway blink or glow or something when it was in view to be AR.

    The speed and GPS are just another data feed, same as the video.

  9. Hey, he is on the ground, the plane is up in the air, he is flying the plane through a camera set up to display information on the plane to him on the ground. It is A/R.

    People don’t like people who play Lawyer-Ball.

  10. AR means it is ‘augmenting’ what you’re really experiencing in ‘reality’, such as superimposing 3d information over your surroundings.

    Despite the fact that it is an awesome build, it is not AR, it is a drone with a head-mounted display.

  11. It’s sad that dedicated graphics chips seem to have disappeared from the market. Lots of hobbyists now have to create their own rendering routines. Anyway, awesome build- very professional.

  12. @mzero – very funny!

    Quote from wikipedia:
    “In the case of Augmented Reality, the augmentation is conventionally in real-time and in semantic context with environmental elements, such as sports scores on TV during a match. ”

    @Marco –
    It’s not that sad – really low end mcus can do all of this, and are very inexpensive. I don’t want to go back to trying to savagely beat video out of old-school graphics chips. Now I just include a few lines of code in my app or use any of the standard text-to-video routines for any MCU around. You can still buy Pic chips and AVRs to do video if you want… it just isn’t efficient anymore.

    Don’t feel bad… I once worried because character generator ROMS had disappeared from the market.

  13. @frozenlazer
    how does that NFL thing work anyway? it looks super clean and even allows for other objects to obscure it, such as players walking over it

    is it automated?

  14. Not AR, just a simple overlay, which in itself, is pretty cool!

    It would be AR if it could recognize certain markers on the ground and display certain information about that.

    Overlaying hud information with integrated data comes nowhere near AR.

  15. He’s viewing the world through the plane’s eyes while standing on the ground. That sounds like a reality augmentation to me.

    I’ve been wanting to do this since I was 13.

  16. So, the HUD in a figher is also AR? What about the overlay in a camcorder?

    Hell, you might as well call my rifle crosshair AR.

    … this isn’t AR.

  17. Imagine if you put that camera in a small bubble and connected it to servos controlled by accelerometers on the head mounted display. he could look around as if he were in a cockpit, and the HUD could counter the movement on the display and stay right up front of the craft.

  18. Hi!

    I am currently working on a university project, trying to build a race car. We are interested in some kind of radio link between the car and the depot, for data transmission during the race. May I ask how you managed to get sufficient range and bit rate to send video from the airplane?

    Tanks!
    Fredrik Bagge
    Lund Institute of Technology

  19. I noticed that the website for this is down. Does Fabien have any intention of putting it back up? Have the powers that be decided that he’s sharing too much engineering knowledge? If anyone has a mirror of the site, I’d love to see it. Pls post here. I have the google cache sites, but the images (and thus schematics) are missing!

    Cheers,

    J

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s