Perforated rolls of paper, called piano rolls, are used to input songs into player pianos. The image above was taken from a YouTube video showing a player piano playing a Gershwin tune called Limehouse Nights. There’s no published sheet music for the song, so [Zulko] decided to use Python to transcribe it.
First off the video was downloaded from YouTube. This video was processed with MoviePy library to create a single image plotting the notes. Using a Fourier Transform, the horizontal spacing between notes was found. This allowed the image to be reduced so that one pixel corresponded with one key.
With that done, each column could be assigned to a specific note on the piano. That takes care of the pitches, but the note duration requires more processing. The Fourier Transform is applied again to determine the length of a quarter note. With this known, the notes can be quantized, and a note duration can be applied to each.
Once the duration and notes are known, it’s time to export sheet music. LilyPond, an open source language for music notation, was used. This converts ASCII text into a sheet music PDF. The final result is a playable score of the piece, which you can watch after the break.
26 thoughts on “Transcribing Piano Rolls With Python”
Bravo! A great hack!
Wow, this is really cool.
Yet another python nazi, same could be done with MATLAB in much less code.
Nazi? I think your understanding of that term is extremely, fundamentally, idiotically broken. Or maybe it’s just you that’s broken.
Nice project BTW, I like reverse engineering and automata of different kinds so it makes me very happy! :)
MATLAB isn’t free. Also, it (really moreso Simulink, but…sigh, we’ve found MATLAB bugs too, particularly in the parallel computing toolbox or whatever the hell it’s called) is the curse of my existence as a NASA contractor.
So…bleh. If you REALLY don’t like Python (I tend to go to Perl for quick stuff), at least use Scilab or something instead of keeping Mathworks alive.
Yet another MATLAB nazi
Vitriolic anti-python commenters are equally as annoying as pro-python commenters. I dislike python, but comments like this make me want to use it exclusively, just to tweak you.
Would you please show me the link for my free (and legal) matlab download? Until then, quiet please.
I feel like there’s some kind of internet meme I’m missing. Was there some sort of major Python-related drama a while back? Did Python’s creator get caught molesting moose?
Or was that just a troll?
Naagh… …I’m pretty sure he wasn’t caught molesting a troll!
How about doing an FFT on the sound track, and restoring the notes from there ?
that is a lot harder than it sounds because you don’t have pure sine waves. harmonics from lower octaves will, by design of the piano, interfere with and appear to be higher octaves as well.
Another problem could be, that the lead pipes for the individual keys in the pianola may be broken, and you cannot get a complete audio recording.
Another wonderful example of “when it comes to music digital was first”. Although 1860 is the first sound recording made thru visual interpolation. Pinned barrels go back half a millennium. There was an experimental device called the melograph. Perhaps there are lines on the back of some papers in libraries that can be converted into music and be attributed to Mozart or Beethoven.
Great hack! It must be good it’s troll worthy.
And for the Nay sayers:
Yes but can you write python code using MATLAB? Then we’ll start to be impressed…
How about Monty Python Nazis?
I was expecting something about direct conversion to midi while he was at it. That sounds straightforward from what was done.
Overall, this is a neat thing. Well done!
This reminds me of a project I’ve been thinking about. I have a number of 80 column Fortran IV punch cards from my earliest computing days and I was thinking of making something to read them with a mechanical feed system to move each one past a column of LEDs and light sensors but maybe I should just use my Raspberry Pi with a camera module, take a succession of stills and process the images to create the original Fortran code.
Why not both? Mechanical feeder passing each card in front of the camera.
Yes, I could use a feed system but it would take a lot longer to make it than the time saved for the few hundred cards I have.
In my original idea with a column of sensors a card transport system is more useful as positioning each card for each column in alignment with the sensor array by hand would be a real pain but with a still photo of each card the ‘scanning’ would be in the software.
What I really like about this is that he couldn’t find any sheet music for it so he converted a piano roll into sheet music with a great hack.
I’m working on a somewhat similar projects, but with scans of perforated metal discs for old music boxes instead of paper tape. The really notable differences with the discs are: more noise, one of the axes are radial and the requirement of accurate center-hole placement. Otherwise it’s much the same.
Much noise can be dealt with using filters, the radial axis can be dealt with using standard geometry, and the centerhole detection might be automated with a bit of shape analysis.
I’d be interested to hear how you go with that. I have a music box disc with a wonderful tune and have not found an arrangement like it anywhere.
(grr, can’t edit posts. I forgot to tick the ‘notify’ check boxes)
It’s working out rather great. The tool I’ve made have been able to have discs manually analyzed for some time (by marking every single needle by clicking at them), but the thing I’m working on which is the goal of the project, is to automatize all of it.
Depending on the disc (in my case Symphonion nr.4/24cm/9.5″ 72-track discs), doing a rough markup manually takes about 2 hours. It takes an additional 1-2 hours to apply fine ajustements so the bass and bright tones hit exactly right. An automated analysis takes less than 2 minutes.
how about some Nancarrow?
Hi there, that was a brilliant video , Can the user/ or player of that piano play this type of music too ?
Please be kind and respectful to help make the comments section excellent. (Comment Policy)