Augmented reality UAV controller

Controlling a long-range unmanned aerial vehicle is much easier if you have an augmented reality system like [Fabien Blanc-Paques] built. On board the aircraft you’ll find a sensor suite and camera, both transmitting data back to the operator. As the title of this post indicates, the display the operator sees is augmented with this data, including altitude, speed, and a variety of super-handy information. For instance, if you get disoriented during a flight there’s an arrow that points back to home. There’s also critical information like how many milliamp-hours have been used so that you can avoid running out of juice, and GPS data that can be used to locate a downed aircraft. Check out some flight video after the break.

Comments

  1. Aero says:

    I couldn’t go back to regular RC planes after that. Impressive!

  2. Remarknl says:

    i dont see any argumented reality.. just a hud..

  3. Super sweet!

  4. brad says:

    i wish my reality was augmented with smooth muzak like that.

  5. turn.self.off says:

    i can see some copter based UAV used by the military as spotters in this way.

  6. dan says:

    That setup turns rc planes from something I find mildly interesting to something I find totally badass. It’s like you’re actually flying! If you had the money to burn you could equip a couple of them with BB-gun cannons for aerial dogfights.

  7. nave.notnilc says:

    it’d be augmented reality if the plane were being used to augment /his/ view, not augment the plane’s view, imo

  8. Slanesch says:

    cool. i can’t wait to see whats next!

  9. JC says:

    FPV flying with Pan/tilt gyro controlled camera makes this seem like child’s play.

  10. Decepticon says:

    One word: Rawesome.

  11. Fellangel80 says:

    put that in the MIG build.

  12. phil says:

    this looks awesome. question though. is there a particular reason he opted for a camera above as opposed to below the plane? is it a functional decision or is it of no consequence?

  13. MS3FGX says:

    Interesting hardware but as already said, this is not AR, simply an on-screen display. AR systems add digital objects to the user’s field of view. This system only displays textual information, and even then, over the video feed from a remote device rather than the first person perspective of the user.

  14. MS3FGX says:

    @phil
    Beyond being a more pilot-like position to view the plane from, putting the camera on top means you can belly-land the plane without risking damage to the camera. These types of planes generally don’t have landing gear, and judging by the way it was sitting at the start of the video, this one is no exception.

  15. cknopp says:

    Why is the prop not spinning in the vid?

  16. Charper says:

    At the risk of beating the “not augmented reality” into the ground…

    You already covered an RC plane with a very similar HUD before. As a bonus it destroyed stuff!

    HAD UAV reigns down vengeance upon thee

    FYI the Youtube link on that one is broken, for ppl wanting to see it:

  17. adam says:

    @cknopp
    Looks like a powered glider (folding prop blades) so it probably has a good enough glide ratio to not need power all the time.

  18. bilbao bob says:

    Even if someone publishes a hack showing how to connect to the cerebral cortex and lets him…

    Well, in my book displaying blocky text most certainly counts as A/R. No, it’s not as cool as using a polhemus unit to control the plane instead of an R/C transmitter, but this is well done and certainly augments reality. His brain was looking out that plane’s virtual cockpit with a HUD – from the ground.

    Could it be cooler, use a power glove and burn a ton of graphics power to make something that looks like a circa 1997 virtual reality project? Sure. But why bother?

  19. bilbao bob says:

    PS – @MSGFX I’m a talkin’ to you:

    >AR systems add digital objects to the user’s field of view.

    Your view of AR is too narrow. I understand that you want this system to include video-game like graphics and Synthetic Vision features before you’ll call it AR, but I beg to differ.

  20. Anon says:

    UAV online comrads!!

  21. Non says:

    Enemy UAV is down!

  22. Muralha says:

    Please, this is not AR!

    If so, any on screen display information would AR, which is not!
    AR has to include real world information with virtual environment information… so where is the virtual???
    GPS… No!
    Distance and orientation to home… “Probably” can be AR.
    Battery power, clock, date… No!

    This is simply: RC with FPV/RPV (first person view/remote pilot view) and with OSD (on screen display).

  23. frozenlazer says:

    To me, the yellow first down line on NFL broadcasts is AR the score board at the top of the screen is not.

    The arrow home is close but really it would need to have the actually image of the runway blink or glow or something when it was in view to be AR.

    The speed and GPS are just another data feed, same as the video.

  24. Greycode says:

    Hey, he is on the ground, the plane is up in the air, he is flying the plane through a camera set up to display information on the plane to him on the ground. It is A/R.

    People don’t like people who play Lawyer-Ball.

  25. Steve says:

    Not augmented reality. A simple OSD. Lots of heli and plane pilots use them these days.

  26. mzero says:

    According to Wikipedia, it is !

    http://en.wikipedia.org/wiki/Augmented_reality

    AR = Live, direct view of physical real world + extra computer-generated imagery elements.

    Augmented reality (AR) is a term for a live direct or indirect view of a physical real-world environment whose elements are augmented by virtual computer-generated imagery.

  27. andar_b says:

    AR means it is ‘augmenting’ what you’re really experiencing in ‘reality’, such as superimposing 3d information over your surroundings.

    Despite the fact that it is an awesome build, it is not AR, it is a drone with a head-mounted display.

  28. Marco says:

    It’s sad that dedicated graphics chips seem to have disappeared from the market. Lots of hobbyists now have to create their own rendering routines. Anyway, awesome build- very professional.

  29. richard mathie says:

    folding propellers that don’t fold :(

  30. bilbao bob says:

    @mzero – very funny!

    Quote from wikipedia:
    “In the case of Augmented Reality, the augmentation is conventionally in real-time and in semantic context with environmental elements, such as sports scores on TV during a match. ”

    @Marco –
    It’s not that sad – really low end mcus can do all of this, and are very inexpensive. I don’t want to go back to trying to savagely beat video out of old-school graphics chips. Now I just include a few lines of code in my app or use any of the standard text-to-video routines for any MCU around. You can still buy Pic chips and AVRs to do video if you want… it just isn’t efficient anymore.

    Don’t feel bad… I once worried because character generator ROMS had disappeared from the market.

  31. bilbao bob says:

    If it helps, any HUD is actually just a subset of AR.

  32. sarsface says:

    I love how the prop revolutions match up with the refresh rate.

  33. sarsface says:

    I’m retarded

  34. flyboy says:

    This isn’t really anything new. Check out http://www.diydrones.com for some excellent tips.

    Also, this is my school. There’s an e-mail link if you’re interested in what was used.

    http://www.nmc.edu/programs/academic-programs/aviation/videos/video-uas-testing.html

  35. octel says:

    @frozenlazer
    how does that NFL thing work anyway? it looks super clean and even allows for other objects to obscure it, such as players walking over it

    is it automated?

  36. Eirinn says:

    Not AR, just a simple overlay, which in itself, is pretty cool!

    It would be AR if it could recognize certain markers on the ground and display certain information about that.

    Overlaying hud information with integrated data comes nowhere near AR.

  37. Volfram says:

    He’s viewing the world through the plane’s eyes while standing on the ground. That sounds like a reality augmentation to me.

    I’ve been wanting to do this since I was 13.

  38. draeath says:

    So, the HUD in a figher is also AR? What about the overlay in a camcorder?

    Hell, you might as well call my rifle crosshair AR.

    … this isn’t AR.

  39. Rizz says:

    Imagine if you put that camera in a small bubble and connected it to servos controlled by accelerometers on the head mounted display. he could look around as if he were in a cockpit, and the HUD could counter the movement on the display and stay right up front of the craft.

  40. Brad Hein says:

    Way to go! This is an awesome project! I was trying to see what wireless protocol is used… 802.11b? I like the sector antenna on the hat :)

  41. walt says:

    cool but lean what AUGMENTED REALITY means. this is a HUD.

  42. Hi!

    I am currently working on a university project, trying to build a race car. We are interested in some kind of radio link between the car and the depot, for data transmission during the race. May I ask how you managed to get sufficient range and bit rate to send video from the airplane?

    Tanks!
    Fredrik Bagge
    Lund Institute of Technology

  43. John says:

    I noticed that the website for this is down. Does Fabien have any intention of putting it back up? Have the powers that be decided that he’s sharing too much engineering knowledge? If anyone has a mirror of the site, I’d love to see it. Pls post here. I have the google cache sites, but the images (and thus schematics) are missing!

    Cheers,

    J

  44. John says:

    Never mind. It seems to be back up and running again.

    Cheers,

    J

  45. DroneGeek says:

    I think it would be better to have a full autopilot such as the Ardupilot Mega Kit as should you loose video or rc link signal your expensive aircraft will return back to you.

    What do you guys think?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 96,513 other followers