We’ve covered plenty of interesting human input devices over the years, but how about an instrument? No, not as a MIDI controller, but to interact with what’s going on-on screen. That’s the job of GuitarPie, a guitar-driven pie menu produced by a group at the University of Stuttgart.
The idea is pretty simple: the computer is listening for one specific note, which cues the pie menu on screen. Options on the pie menu can be selected by playing notes on adjacent strings and frets. (Check it out in action in the video embedded below). This is obviously best for guitar players, and has been built into a tablature program they’re calling TabCTRL. For those not in the loop, tablature, also known as tabs, is an instrument-specific notation system for stringed instruments that’s quite popular with guitar players. So TabCTRL is a music-learning program, that shows how to play a given song.
With this pairing, you can rock out to the tablature, the guitarist need never take their hands off the frets. You might be wondering “how isn’t the menu triggered during regular play”? Well, the boffins at Stuttgart thought of that– in TabCTRL, the menu is locked out while play mode is active. (It keeps track of tempo for you, too, highlighting the current musical phrase.) A moment’s silence (say, after you made a mistake and want to restart the song) stops play mode and you can then activate the menu. It’s well a well-thought-out UI. It’s also open source, with all the code going up on GitHub by the end of October.
The neat thing is that this is pure software; it will work with any unmodified guitar and computer. You only need a microphone in front of the amp to pick up the notes. One could, of course, use voice control– we’ve seen no shortage of hacks with that–but that’s decidedly less fun. Purists can comfort themselves that at least this time the computer interface is a real guitar, and not a guitar-shaped MIDI controller.
Wonderful, please expand this to other instruments that support tablature (diatonic accordeon would be a good start)
Wow really cool. A great use of DSP!
That’s a very neat hack and genuinely useful! Getting fancy, if you get the audio from your signal chain with a footswitch option to mute the amp, a live performer can use this to switch backing tracks easily, etc.
of course! because the ‘guitar hero’ game needs a UI
i’ve been dreaming for a long time about making this sort of thing as a UI for non-music apps. it’d be epicly impractical (even if i lost the use of my hands, speech recognition has more bandwidth than tone recognition). but it’d be a good exercise to improve my ear. but i keep getting hung up on questions like
absolute notes (i can sing two octaves, or about 24 semitones hah) or relative intervals?
trigger on tone detection, or provide feedback until the end of the tone so i can ‘hunt’ for the right note and only stop when i’ve hit it?
pie menu or ??something else?? 24 tones isn’t quite an alphabet, and 24 tones is already an ambitious goal.
really the point of the thing is how incredibly difficult it would be to get anything done — the difficulty is training! so i guess the details don’t matter, but on the other hand i’ve already made a number of tools just for training (with no “input device” aspect) so maybe i’m just being silly. it’s a dream :)
I just want to hear them use this to type a letter!
This is what UI people should be doing instead of changing colors and using the same 50-year-old radio buttons and checkboxes. The world is due for a whole new set of input methods.