[Horiken Engineering], which is made up of engineering students at the department of aerospace at the University of Tokyo have developed an autonomous quadcopter that requires no external control — and its tiny. By using two cameras and a sonar sensor, the quadcopter is capable of flying by itself due to its ability to process the data from the on-board sensors. To do the complex data processing fast enough to fly, it is using a Cortex-M4 MCU, a Spartan-6 FPGA, and 64MBs of DDRSDRAM. It also has the normal parts of a quadcopter, plus gyros, a 3D printed frame and a 3-axis compass. The following video demonstrates the quadcopter’s tracking ability above a static image (or a way point). The data you see in real-time is only the flight log, as the quadcopter receives no signal — it can only transmit data.
Is this the first step towards Amazon’s fleet of package delivering drones? It’s certainly going to be interesting when quadcopters are a common occurrence in public…
Woah monocular realtime camera that can do 3DOF or 6DOF tracking? On a microcontroller? What sorcery is this?
This one’s got 2 cameras and a Cortex M4, so I dunno.
only 1 camera for 3d, the other one seems to be pointed forward and not connected to fpga (probably sends video over RF)
I’d bet good money that’s what the FPGA is for, just processing the video. As for 6DOF, it’s affine transforms of a known marker. The scale of the marker is roughly distance, rotation is roughly heading, etc. It’s actually a bit fiddlier than that, but nothing that we hasn’t been done before. Still, the build as a whole is impressive. Well done.
I don’t think they know the marker beforehand
you dont have to
anything can be a marker, problems start when you have dynamic background, now all of a sudden markers start to move independently and you get lost
http://en.wikipedia.org/wiki/Feature_extraction
simplest and ?most popular? seem to be corner detection, plenty of corners around, we meatbags do love geometry
http://www.robots.ox.ac.uk/~gk/PTAM/
http://robotics.dei.unipd.it/~pretto/index.php?mod=01_Research
http://www.robots.ox.ac.uk/ActiveVision/Research/Projects/summaryindex.html
So does it use something like SIFT features to detect it’s spacial orientation and distance ? What if the surface below it isn’t flat or changes position and/or shape?
The algo only works with a stationary scene below. SIFT and SURF are the kind of thing used for consistent feature detection. THere is an opensource, patent unencumbered feature detector inside the Hugin project. Which is pretty neat all on its own…
Electronic skeet . . .
This is what the future holds. We will electronically hunt drones with lazer rifles, and the stray light will be collected and recycled…if possible.
As an ME student trying to get into autonomous vehicles, I’m *dying* to see their algorithms.
my guess would be something along the lines of using the range finder to estimate the size of the “bar code” on the ground then converting it to a vector image. searching that vector image for black squares that are in the know pattern. after that checking the orientation of the “bar code” this would give retentive position using some basic geometry.
But that is a unqualified guess.
its a bad guess
3D is reconstructed using Structure from Motion algorithm
http://www.morethantechnical.com/2012/02/07/structure-from-motion-and-3d-reconstruction-on-the-easy-in-opencv-2-3-w-code/
This is one of the ways how humans estimate 3D when using one eye – parallel head micro movements enable us to reconstruct 3d from that motion. ( Motion parallax )
http://en.wikipedia.org/wiki/Depth_perception
So there is no need for a specific bar code or is that just to be a reference point ?
no need for special markers
later in the video they have something that looks like newspapers under the drone, just to give it texture for good features
It could be that they have the feature extraction handled by the fpga and the rest of the monoSLAM computation can then be done by the arm.
If I understand correctly, image feature extraction on a spartan is available as a package you can download (buy?):
http://www.xilinx.com/products/intellectual-property/EF-DI-VID-OBJ-SEG.htm
One buckshot round and it’s fancy control systems won’t be of much use.
Not much of a criticism. The same could probably be said of you!
Quoting Dr. Leonard “Bones” McCoy – “fantastic machine ! no OFF switch !!”
“Autonomous?” How does it decide what it wants to do?
if anyone wants to have a go
http://www.robots.ox.ac.uk/~gk/PTAM/
https://www.youtube.com/watch?v=E35xbo3r8rA
Looks like guys at Horiken Engineering managed to squeeze PTAM on an FPGA. Very impressive. Would love to be able to buy their FPGA alone, or see the code ( sadly Japanese arent exactly known for open sourcing).
Its able to see how its moving relative to the B&W pattern. If it loses its way as it moves (after buffer fills up wih many more samples) – then it will be globally lost but still able to orient itself locally as long as there is contrast in the visual pattern it can see below.
If you seriously are interested in this – then here’s a python based course (incl code) and video set. Its the full SLAM with bundle adjustments etc and pretty much state of the art. Implement in your embedded CPU of choice. http://www.youtube.com/playlist?list=PLpUPoM7Rgzi_7YWn14Va2FODh7LzADBSm
Thanks Claus Brenner.
I’d suggest the platforms polyhelo or openPilot revolution – both of which use Cortex M4 ARM cpus with FPU. They have many dof sensors on them.
When micropython is up and running (Feb maybe?) then you might be able to use an STM32F429 (25 USD from STM) and code it all in python directly. (FYI micropython compiles down to native C for speed).
Behold! The fleet of terminator drones are being released by SKYNET!! To destroy all humans! everywhere on this planet!
Just use one of these: http://www.youtube.com/watch?v=zkBnFPBV3f0&feature=player_embedded
supplied by one of these: https://www.youtube.com/watch?v=ln_jaXAxsgs
And equip your militia with some of these: https://www.youtube.com/watch?v=E8niD-n0lHk