Virtual reality holds the promise of an immersive experience that can satisfy our senses to a level comparable with… well, reality. The field has come a long way, but Sarah Vollmer makes a good point that many of the VR systems currently in use are bulky and difficult to transfer from person to person.
While headsets have become smaller and lighter and now feature improved motion tracking and resolution, their ability to affect the user’s other senses hasn’t seen nearly the same advancement. Haptic feedback systems need to catch up with headsets, and how to unobtrusively allow users feel simulated physical contact in VR is an area Sarah is researching as part of her PhD work. This is the topic of her 2019 Hackaday Superconference talk which you’ll find embedded below.
Controlling the Internet of Things is all about passing information around. Realistically, it doesn’t matter what is used, be it MQTT, HTTP, serial data, whatever, and it doesn’t really matter what data is sent as long as the sender and receiver agree on what the data means. MIDI could be used to pass information back and forth, for example and while MIDI is good for some things, Open Sound Control is a more modern alternative and one area where OSC excels over MIDI is Internet connectivity. [Matt] used OSC to control the lighting he installed in his kitchen.
[Matt] had moved in to a new house and wanted some under-cupboard lighting for his kitchen. He got a few cheap warm white LED lights from the Internet and went about wiring them together. For the controller, an ESP8266-1 was used as well as a 12 volt constant-current buck converter. The software runs on the Sming framework, rather than the Arduino framework, and listens for incoming OSC messages. When it receives a command on a specific channel, a callback function turns the lights on and off. [Matt] also added a switch on the outside of the control box to manually turn the lights on and off.
OSC might not be the right choice for this project, and even [Matt] doesn’t know why he used it, but [Matt] got it working and uses an app on his phone to control it. If he wanted to, he could have used Ableton or another controller to control the lights. (He hasn’t wanted to yet.) OSC is an interesting alternative to MIDI and can also be used with an Arduino without an ethernet shield, or with RFID tags.
The folks at [Design I/O] have come up with a way for you to play the world’s tiniest violin by rubbing your fingers together and actually have it play a violin sound. For those who don’t know, when you want to express mock sympathy for someone’s complaints you can rub your thumb and index finger together and say “You hear that? It’s the world’s smallest violin and it’s playing just for you”, except that now they can actually hear the violin, while your gestures control the volume and playback.
[Design I/O] combined a few technologies to accomplish this. The first is Google’s Project Soli, a tiny radar on a chip. Project Soli’s goal is to do away with physical controls by using a miniature radar for doing touchless gesture interactions. Sliding your thumb across the side of your outstretched index finger, for example, can be interpreted as moving a slider to change the numerical value of something, perhaps turning up the air conditioner in your car. Check out Google’s cool demo video of their radar and gestures below.
Project Soli’s radar is the input side for this other intriguing technology: the Wekinator, a free open source machine learning software intended for artists and musicians. The examples on their website paint an exciting picture. You give Wekinator inputs and outputs and then tell it to train its model.
The output side in this case is violin music. The input is whatever the radar detects. Wekinator does the heavy lifting for you, just give it input like radar monitored finger movements, and it’ll learn your chosen gestures and perform the appropriately trained output.
[Design I/O] is likely doing more than just using Wekinator’s front end as they’re also using openFrameworks, an open source C++ toolkit. Also interesting with Wekinator is their use of the Open Sound Control (OSC) protocol for communicating over the network to get its inputs and outputs. You can see [Design I/O]’s end result demonstrated in the video below.
[Marcus] wrote a guide to using TouchOSC to control your projects. He sent a link to us after reading our feature about using Open Sound Control for Arduino without an Ethernet shield. He’s been using that method for quite some time now, but takes it one step further by using a smartphone as a control device. He designs his own user interface for the iPhone using TouchOSC. This is a package which we’ve seen in other projects but now you can get an idea of how easy it really is.
The project starts by interfacing an Arduino with the device you’d like to control. The circuit above patches into a remote control using a couple of transistors. Now the Arduino can simulate button presses on that remote, sending the signal to turn a light on or off. Next, TouchOSC is used for the smartphone – here it’s an iPhone but the suite works on Android as well. In the video after the break you can watch a quick interface design demo. Buttons are dragged into existence, uploaded to the phone, then configured to control you device over a network. A Processing sketch listens for OSC commands and then sends instructions to the Arduino via USB.
Open Sound Control (OSC) is a communications protocol that can be seen as a modern alternative to MIDI. It’s specifically designed to play nicely with network communication systems. The problem with using it along with Arduino-based gadgets is that you then need to use something like an Ethernet shield to provide the network connection. [Liam Lacey] decided to use Processing as a go-between for OSC and the Arduino in lieu of the Ethernet Shield. One of the major benefits of this method is that it gives you some flexibility when it comes to how the Arduino communicates. Since a USB connected Arduino can be addressed by the Processing sketch in the same way as an AVR chip connected via an RS232 serial port, [Liam’s] method will allow you to prototype on an Arduino board, but transition to your own non-USB hardware for the finished project. The one big drawback to this method is the need to have a computer connected to your controller, but we’d bet you’d need one to run MAXmsp anyway.
There was a time when a drummer would grab some sticks and lay out a groove using the items around him as instruments. [Lsa Wilson] would rather not work quite that hard and has chosen to do the same thing by tapping on an iPhone screen. As you can see in the clip after the break, many of the items in the room around him have been fitted with solenoids. Each is connected to an Arduino which is then controlled by Open Sound Control and interfaced with the iPhone via TouchOSC. We love the sounds being created and can’t help being reminded of the Multixylophoniomnibus.
Have you been working on a MIDI controller that uses RFID to identify and control different instruments? No? Neither have we but now we’re going to have to look into it. That’s because [Martin.K] has done a lot of the work for us. His nfOSC package links an RFID reader to the Open Sound Control library.
In the video after the break we see [Martin] placing RFID tags onto a Touchatag reader. With each addition, his software triggers a tag add event that OSC picks up and translates to a midi event; in this case it adds a new instrument to the mix. Can this be used to relieve musicians from staring at computer screens during performances? What if there was a small shelf in front of you? As you happily play your electric Didgeridoo, small items with RFID tags on them can be added or removed from that shelf to change the samples that are triggered when toiling away on that sonic weapon. This should be fun!