For a final design project, [Frank] and his group took on an augmented reality project. The goal was to make objects interactively controllable by pointing a smartphone at them. Their solution was Augmented Reality Universal Controller and Identifier (ARUCI).
The system locates controllable objects by sensing IR beacons that contain identifiers for each object. The IR is received by a Wiimote sensor, which has been integrated into a custom PCB. This board sits in a 3D printed enclosure, and mounts to the back of a smartphone. The electronics are powered by tapping off of the phone’s battery.
Commands are sent to devices using a custom 2.4 GHz protocol which was implemented using the ATmega128RFA1. Each device has another ATmega to receive the signal and control the real world object. In their demo, the group shows the system controlling devices including a TV, a radio, and an RC car.
The system provides an interesting way to interact with objects, and the hardware integration is quite impressive. After the break, watch [Frank] give a demo.
12 thoughts on “IR Based Augmented Reality”
My favorite part
Definitely an interesting way to interact with objects. I would feel like Tony Stark in Iron Man 2, controlling those TV’s at the press conference.
This seems much more complicated than necessary. Why not just use a QR-Code in conjunction with the phones camera (and embedd a wifi/bt module in each device)?
@Max re: QR code
IR has much longer range and can be hidden. A QR code would be big and ugly ro work at long ranges.
Why not just tape an ir-pass filter over the phone’s camera and use OpenCV?
Wouldn’t be very useful then, do you expect to take the tape off whenever you don’t want to use it, and tape it back on when you do?
Or just use the Camera “as is” and recognise the leds based on the (pretty) unique color IR appears on loosely filtered cameras anyway (if you ever pointed a camera at an ir-remote you know what i mean)
The visibility of the IR after the filter is not very bright, the colour is not unique, it’s slightly purple
The image gets a purple shade if the IR filter is removed.
I still need my phone as a phone, so no invasive operations were desired.
I didn’t want to risk ruining my phone, or its camera by doing an invasive hack. Risk management is important when dealing with limited budget and tight deadlines.
Without the filter, the image would have a shade of purple.
It would be great if they made CMOS or CCD cameras with a separate IR channel which would solve a lot of problems, we would have ditched the smartphone and went with a 32 bit microcontroller plus our own colour LCD instead.
The IR filter on your camera keeps out the IR component to which the sensor is unavoidably sensitive. The inaccurately named “IR filter” placed in front of the IR LEDs is transparent to IR and opaque to visible light frequencies. This later filter is very useful if you want to weed out false triggers.
Pixel density is so outrageous today that camera makers are beginning to implement HDR direct and/or more than 3 tone colors. I wouldn’t be at all surprised to see cameras soon sporting wide spectrum imaging options.
I’m really curious how you determine which object in the Wiimote’s field of view corresponds with what the single IR receiver is decoding? (Theories me and my friend posited include time-devision multiplexing on each device, or a relatively narrow field of view for the IR receiver)
Please be kind and respectful to help make the comments section excellent. (Comment Policy)