Microsoft’s New Simulator Helps Train Drone AIs

Testing any kind of project in the real world is expensive. You have to haul people and equipment around, which costs money, and if you break anything, you have to pay for that too! Simulation tends to come first. Making mistakes in a simulation is much cheaper, and the lessons learned can later be verified in the real world. If you want to learn to fly a quadcopter, the best thing to do is get some time behind the sticks of a simulator before you even purchase anything with physical whirly blades.

Oddly enough, the same goes for AI. Microsoft built a simulation product to aid the development of artificial intelligence systems for drones by the name of Project AirSim. It aims to provide a comprehensive environment for the testing of drone AI systems, making development faster, cheaper, and more practical.

The Cheaper Way of Making Mistakes

Many companies are developing drones to take on tasks that have historically been executed manually by humans. For example, lofting a drone up to look at a wind turbine nacelle or insulators on a power line is far safer than having a human climb up the structure. However, with a basic drone, this requires a skilled pilot to avoid crashing into expensive hardware, particularly if conditions get hairy. Automation would allow drones to handle their tasks themselves at the mere push of a button.

AI models can quickly learn through trial and error in a simulator. Credit: Microsoft

Testing drone AIs in the field is difficult, though, with plenty of risks involved. Let’s say you’re building an AI-controlled drone to inspect powerlines. If the drone’s navigation algorithm isn’t quite right, you might watch it sail out over the ocean, never to return. Meanwhile, if you haven’t got the object avoidance quite right, it could run straight into the powerlines it’s supposed to be inspecting. The consequences are even worse if it happens to take down power to the local area in the process.

Simulation offers a way to test drone AIs in various conditions without the risk to life, limb, and property. Drone behaviour can be quickly run through a sanity check, and silly mistakes can be spotted easily. If the drone crashes in the simulator, you simply hit reset, make some changes, and go again. In real life, you have to pick up the pieces and rebuild the drone first, after apologizing to the owner of whatever it just hit.

Microsoft’s Offering

Project AirSim is Microsoft’s new simulation offering for drone development. It enables drone AIs to be tested and developed in a realistic 3D world. Conditions like wind, rain, and snow can all be simulated, along with different environments, from urban cityscapes to thick forests.

Included will be recreations of places like New York City and London, as well as more generic offerings such as a virtual airport. Data from Bing Maps will also be available to create detailed 3D environments of other locations around the world.

Having such a environment to test in is invaluable for developing not only AI to control drones, but the sensors those systems rely on. For example, when developing a machine vision system, one may wish to test how it works during the day, at night, and in dense foggy conditions. Achieving this in reality would require waiting for the weather to change, but it can be simulated at will.

Project AirSim also hopes to help designers quantify the performance of their drones. A particularly windy test environment could show how battery life is affected in such conditions. Things like climb rates and loiter times can also be quantified in the simulator to collect ballpark figures ahead of real-world testing.

Recreations of real-world locations will serve as proving grounds for drones in the simulator. Credit: Microsoft

Project AirSim is a new offering, developed from Microsoft’s earlier open-source project of the same name. That codebase is still available to the community, but required users to have significant experience in coding and machine learning disciplines.

To open up development to a broader audience, Project AirSim will include AI building blocks for common control tasks. Users will be able to integrate existing AI systems for tasks like automatic landings or obstacle avoidance into their drones. Having these modules included makes sense, as the vast majority of drones can benefit from the use of similar modules applicable to many missions.

Some heavy-duty collaborators are working with Microsoft on the project, too. Ansys has helped develop physics-based sensor simulations to provide realistic sensor outputs to simulated drone control systems. Meanwhile, MathWorks is working on a system to let users import their own physics models to the simulator via its Simulink software.

Drones are set to revolutionize all kinds of fields, from environmental monitoring to delivering the mail. Simulation will go a long way to helping develop autonomous drones as quickly and affordably as possible. Microsoft’s new offering will likely find plenty of customers eager to get to testing.


15 thoughts on “Microsoft’s New Simulator Helps Train Drone AIs

  1. The few times I used M$ Flight Simulator, I crashed relatively quickly.
    That left me wondering if it 🤔 was a disconnect between real life (seat of the Pants) and the simulation, or that the same thing would happen in a real situation.
    BTW, I am aware that seat of the Pants flying should not be trusted in IFR.

      1. I don’t remember, but being able to run MSFS in DOS with sound and all the goodies (XMS, mouse, joystick, etc.) was the final exam I gave when I taught Intro to Operating Systems about 2 decades ago

    1. The new version let’s you adjust the laws of physics or rules so it can be more like a regular game or you could use it to actually learn how to fly it’s so crazy accurate. It’s also absolutely gorgeous… After the first year major update to get it running on console it also basically doubled PC performance so I could run it at 4k60.

      Just wish it supported DLSS 2.x or FSR 2.x for an extra boost in performance.

  2. “Conditions like wind, rain, and snow can all be simulated, along with different environments, from urban cityscapes to thick forests.”

    Wonder how well this simulates air dynamics close to buildings and other structures.

  3. We mostly use XPlane for our simulations, which has limited support to be plugged into Ardupilot. I’m interested in seeing what comes of this, but from the way the article is written it seems to be another deliberate anti-documentation commercial fork of open source. We’ve been down that road before, and neither project survives.

    1. Embrace, extend, extinguish!

      Sucks to be the open-source project, losing a big sponsor and having them take all the work for themselves.

      I honestly haven’t been super pro-GPL vs MIT or BSD or Apache, but seeing this kind of behavior reminds me why the GPL-style copyrights exist.

  4. Ah, training AI on a simulated model! Two things that work terribly combined into one! At least flying in the open air is a pretty easy thing for computers to do competently. I feel sorry for people who are still working on autonomous cars. Have fun beating your head against that problem for four more decades.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.