Stanislaw playing notes on his MIDI keyboard, with the LEDs on the LED strip lighting up right above the note he's playing, driven by the Raspberry Pi that runs a script based on the Pianolizer toolkit

Pianolizer Helps Your Musical Projects Distinguish Notes

April 26, 2022 by Arya Voronova 9 Comments

[Stanislaw Pusep] has gifted us with the Pianolizer project – an easy-to-use toolkit for music exploration and visualization, an audio spectrum analyzer helping you turn sounds into piano notes. You can run his toolkit on a variety of different devices, from Raspberry Pi and PCs, to any browser-equipped device including smartphones, and use its note output however your heart desires. To show off his toolkit in action, he set it up on a Raspberry Pi, with Python code taking the note data and sending color information to the LED strip, displaying the notes in real time as he plays them on a MIDI keyboard! He also created a browser version that you can use with a microphone input or an audio file of your choosing, so you only need to open a webpage to play with this toolkit’s capabilities.

He took time to make sure you can build your projects with this toolkit’s help, providing usage instructions with command-line and Python examples, and even shared all the code used in the making of the demonstration video. Thanks to everything that he’s shared, now you can add piano note recognition to any project of yours! Pianolizer is a self-contained library implemented in JavaScript and C++ (which in turn compiles into WebAssembly), and the examples show how it can be used from Python or some other language.

[Stanislaw] also documented the principles behind the code, explaining how the note recognition does its magic in simple terms, yet giving many insights. We are used to Fast Fourier Transform (FFT) being our go-to approach for spectral analysis, aka, recognizing different frequencies in a stream of data. However, a general-purpose FFT algorithm is not as good for musical notes, since intervals between note frequencies become wider as frequency increases, and you need to do more work to distinguish the notes. In this toolkit, he used a Sliding Discrete Fourier Transform (SDFT) algorithm, and explains to us how he derived the parameters for it from musical note frequencies. In the end of the documentation, he also gives you a lot of useful references if you would like to explore this topic further!

What are you going to build with this? Maybe, a box that records you playing the flute and instantly turns it into sheet music? Or, perhaps, an AI that continues the song for you when you stop?

Continue reading “Pianolizer Helps Your Musical Projects Distinguish Notes” →

Turning Sounds From A Flute Into Sheet Music

December 29, 2019 by Sharon Lin 12 Comments

Composing music can be quite difficult – after all, you have to keep in mind all of the elements of musical theory, from time signature and key signature to the correct length for all of the notes. A team of students from Cornell University’s Designing with Microcontrollers class developed a solution for this problem by transcribing sounds from a flute into sheet music.

The project doesn’t simply detect the notes played – it is able to convert the raw audio into a standardized music score complete with accurate note timings and beats per minute. Before transcribing the music, some audio processing was necessary. The team chose to use a Sallen-Key filter to amplify the raw audio input due to its complex conjugate poles. They then used a fast Fourier Transform (FFT) to determine the frequency for the input note, converting the signal from the time domain to the frequency domain.

The algorithm samples the data to generate an input signal, using the ADC on the microcontroller to receive input from the microphone. It takes the real and imaginary components of the sampled signals and outputs a pair of real and imaginary amplitude components corresponding to the sampled frequency, evenly spaced from 0 to the Nyquist rate (half the sampling rate). The spacing of these bins and the bin with the largest amplitude are used to convert the signal back to a real frequency and a MIDI note.

The system uses a PIC32 for the logic. The circuitry for the microphone amplification uses a non-inverting op-amp with a gain of 50 to increase the microphone output signal amplitude from 15 mV to 750 mV to use by the microcontroller’s ADC. The signal is then sent to the anti-aliasing Sallen-Key filter, with a pole at 2.5 kHz and a Q of 1. The frequency was chosen since the FFT samples at 8 kHz and the frequency corresponds to a note out of the range of a flute. As for the filters, only the low pass filter was implemented in hardware. While a bandpass filter could have been implemented in hardware, the team decided on a cleaner software approach.

The project is well-documented on the team’s project page, and it’s certainly worth checking out for more detailed discussions on the keypad controls and the software side of the audio processing. If you want to learn more about the FFT, check out this 2016 Hackaday Prize entry for an FFT spectrum analyezer.

Continue reading “Turning Sounds From A Flute Into Sheet Music” →