Visitors to the Wizarding World of Harry Potter at Universal Studios are able to cast “spells” by waving special interactive wands in the air. Hackers like us understand that there must be some unknown machinations happening behind the scenes to detect how the wands are moving, but for the kids wielding them, it might as well be real magic. So when his son asked to have a Harry Potter themed birthday party, [Adam Thole] decided to try recreating the system used at Universal Studios in his own home.
The basic idea is that each wand has a reflector in the tip, which coupled with strong IR illumination makes them glow on camera. This allows for easy gesture recognition using computer vision techniques, all without any active components in the wand itself.
[Adam] notes that you can actually buy the official interactive wands from the Universal Studios online store, and they’d even work with his system, but at $50 USD each they were too expensive to distribute to the guests at the birthday party. His solution was to simply 3D print the wands and put a bit of white prismatic reflective tape on the ends.
With the wands out of the way, he turned his attention to the IR imaging side of the system. His final design is a very impressive 3D printed unit which includes four IR illuminators, a Raspberry Pi Zero with the NoIR camera module. [Adam] notes that his software setup specifically locks the camera at 41 FPS, as that triggers it to use a reduced field of view by essentially “zooming in” on the image. If you don’t request a FPS higher than 40, the camera will deliver a wider image which didn’t have any advantage in this particular project.
The last part of the project was taking the video stream from his IR camera and processing it to detect the bright glow of a wand’s tip. For each frame of the video the background is first removed and then any remaining pixel that doesn’t exceed a set brightness level if ignored. The end result is an isolated point of light representing the tip of the wand, which can be fed into Open CV’s optical flow function to show [Adam] what shape the user was trying to make. From there, his software just needs to match the shape with one of the stock “spells”, and execute the appropriate function (such as changing the color of the lights in the room) with Home Assistant.
Overall, it’s an exceptionally well designed system considering the goal was simply to entertain a group of children for a few hours. We almost feel bad for the other parents in the neighborhood; it’s going to take more than a piñata to impress these kids after [Adam] had them conjuring the Dark Arts at his son’s party.
It turns out there’s considerable overlap between hacker types and those who would like to have magic powers (go figure). [Jennifer Wang] presented her IMU-based magic wand research at the 2018 Hackaday Superconference, and in the past we’ve even seen other wand controlled light systems. If you go all the way back to 2009, we even saw some Disney-funded research into interactive wand attractions for their parks, which seems particularly prescient today.
Isolating IR
The old fashioned method of detecting IR and not visible light, is to put a piece of exposed film negative in front of the camera’s sensor.
Would this be a worthwhile simplification by removing the need for BackgroundSubtractorMOG2?
I actually used a real glass IR filter in the project for the same effect. You can see it in the picture above resting on a white microfiber cloth. I’ve heard that you could use a piece of exposed film negative, but I preferred to use an actual glass IR filter.
Even with the filter it doesn’t negate the need for the background subtractor in all cases. If there is anything particularly reflective within the FOV of the camera, it would potentially be seen as a wand tip. The IR filter doesn’t help in this case, as we are shining IR light out, so any reflector will look like a bright white spot. In my office where I did the majority of the testing I have a bike on the wall, and the wheel reflector caused issues until the background subtractor was added. Another example would be someone sitting on the couch with a reflective patch on their shirt. That said, in the video above the conditions are controlled enough that the background subtractor would not have been required.
MagiQuest wands are $25 and IIRC have active IR xmitters in them.
Still a cool $400 by my count. If you’ve got cash like that to burn on the _favors_ for a kid’s party, you might as well hire a Harry Potter impersonator to come handle things while you and Babs take the yacht out.
LOL and good point old boy, $25 ea adds up when talking about a kids bday party.
there is a William Osman where he puts a retro reflector on a hot dog to show the principal. So you could bring the wand cost down a lot but plastic might hit the sweet spot for durability, safety, and ability to wipe it down with Lysol.
The important part
per wand print price: $25
raspberry pi and camera’s: $100 ish
Setting the bar impossibly high for your friends circle: priceless
I’m a little surprised he 3D printed the wands when he could have taken a page from Tom Sawyer and convinced the kids that making their own with a pruning shears would have been more fun. Just make sure they don’t get into any fights over a black elder tree, since in the Potterverse, those disputes seem to turn lethal pretty quickly.
This is really cool. I’m very impressed. I’ve spent a little time trying to work out how to do this without IR. The best I can come up with is multiple cameras isolating the wand from the person, and trying to identify the tip. I’m sure it’s possible, but my son is only 14 months and hasn’t yet requested a Harry Potter Party. :)
This is really cool. I’m very impressed. I’ve spent a little time trying to work out how to do this without IR. The best I can come up with is multiple cameras isolating the wand from the person, and trying to identify the tip. I’m sure it’s possible, but my son is only 14 months and hasn’t yet requested a Harry Potter Party. :)
Im still waiting for my microsoft wand :(
(was shown by microsoft labs in at least 2010, or even earlier, let you do exactly this, control lights and appliances by pointing at them and gesturing, tried to find link but apparently microsoft acquired some company called wand labs so now all i get is search results about that :P)
And ofc i find it 2minutes after posting, was from ‘microsoft research’, not ‘microsoft labs’
https://www.microsoft.com/en-us/research/publication/xwand-ui-intelligent-spaces/