Guitar Hero was a cultural phenomenon a little over a decade ago, and showed that there was a real fun time to be had playing a virtual instrument on a controller. There are several other similar games available now for different instruments, including one called Trombone Champ that [Hung Truong] is a fan of which replaces the traditional guitar with a trombone. The sliding action of a trombone is significantly different than the frets of a guitar, making it a unique challenge in a video game. But an extra challenge is building a controller for the game that works by playing a real trombone.
Unlike a guitar which can easily map finger positions to buttons, mapping a more analog instrument like a trombone with its continuous slide to a digital space is a little harder. The approach here was to use an ESP32 and program it to send mouse inputs to a computer. First, an air pressure sensor was added to the bell of the trombone, so that when air is passing through it a mouse click is registered, which tells the computer that a note is currently being played. Second, a mouse position is generated by the position of the slide by using a time-of-flight sensor, also mounted to the bell. The ESP32 sends these mouse signals to the computer which are then used as inputs for the game.
While [Hung Truong] found that his sensors were not of the highest quality, he did find the latency of the control interface, and the control interface itself, to be relatively successful. With some tuning of the sensors he figures that this could be a much more effective device than the current prototype. If you’re wondering if the guitar hero equivalent exists or not, take a look at this classic hack from ’09.
