If you’ve put in all the necessary practice to learn bike tricks, you’d probably like an appropriately dramatic soundtrack to accompany your stunts. A team of students working on a capstone project at the University of Washington took this natural desire a step further with the Music Bike, a system that generates adaptive music in response to the bike’s motion.
The Music Bike has a set of sensors controlled by an ESP32-S3 mounted beneath the bike seat. The ESP32 transmits the data it collects over BLE to an Android app, which in turn uses the FMOD Studio adaptive sound engine to generate the music played. An MPU9250 IMU collects most position and motion data, supplemented by a hall effect sensor which tracks wheel speed and direction of rotation.
When the Android app receives sensor data, it performs some processing to detect the bike’s actions, then uses these to control FMOD’s output. The students tried using machine learning to detect bike tricks, but had trouble with latency and accuracy, so they switched to a threshold classifier. They were eventually able to detect jumps, 180-degree spins, forward and reverse motion, and wheelies. FMOD uses this information to modify music pitch, alter instrument layering, and change the track. The students gave an impressive in-class demonstration of the system in the video below (the demonstration begins at 4:30).
Surprisingly enough, this isn’t the first music-producing bike we’ve featured here. We’ve also seen a music-reactive bike lighting system.
Thanks to [Blake Hannaford] for the tip!
This is groundbreaking in many ways, at least for me. Just the idea of having an interactive soundtrack that reacts to what you’re currently doing could be a whole new thing – and, of course, an excuse to have your mobile phone record your complete life. But I could imagine somebody like Brian Eno developing (composing?) a library of interactive musical backgrounds that adapt to the content and flow of your Powerpoint presentations (okay, he probably wouldn’t do something so mundane but I’m sure he’d like the idea).
Several years ago I have predicted Interactive Movies. I guess it’s coming soon. Imagine, you watch a movie like Terminator and you need to assign roles. You assign your friend and yourself. Then you just watch and you can change the way it plays in real time. Some sort of combination interactive Game and Movie.
Like they did with Bandersnatch on Netflix in 2018?
https://en.m.wikipedia.org/wiki/Black_Mirror:_Bandersnatch
There was one shown at Expo 67 in Montreal. The plot forked in different places. The audience voted on which fork to take. https://en.wikipedia.org/wiki/Kinoautomat
I think they tried that in the LaserDisc days. Mysteries with different outcomes. I predated your prediction on people living in a “movie” decades ago at the start of VR.
i think it can be much funner if it works like midi input and generate midi music from samples
I see way too long of time for the strike hold, I assume over strike is what actually hits the tone tongue but re-strike is hindered by solenoid being held on. The strike time should be very short. Solenoids? I have an a back burner project, a soft striker on a piece of spring wire a finger’s length attached to a hard drive arm magnet assembly. It’s ready. Mount on drum frame with a 555 and a transistor could make a tempo drum.