Transcribing Piano Rolls with Python

Piano Roll

 

Perforated rolls of paper, called piano rolls, are used to input songs into player pianos. The image above was taken from a YouTube video showing a player piano playing a Gershwin tune called Limehouse Nights. There’s no published sheet music for the song, so [Zulko] decided to use Python to transcribe it.

First off the video was downloaded from YouTube. This video was processed with MoviePy library to create a single image plotting the notes. Using a Fourier Transform, the horizontal spacing between notes was found. This allowed the image to be reduced so that one pixel corresponded with one key.

With that done, each column could be assigned to a specific note on the piano. That takes care of the pitches, but the note duration requires more processing. The Fourier Transform is applied again to determine the length of a quarter note. With this known, the notes can be quantized, and a note duration can be applied to each.

Once the duration and notes are known, it’s time to export sheet music. LilyPond, an open source language for music notation, was used. This converts ASCII text into a sheet music PDF. The final result is a playable score of the piece, which you can watch after the break.

Comments

  1. Jacques says:

    Bravo! A great hack!

  2. Wow, this is really cool.

  3. agejio says:

    Yet another python nazi, same could be done with MATLAB in much less code.

  4. Trui says:

    How about doing an FFT on the sound track, and restoring the notes from there ?

    • Willrandship says:

      that is a lot harder than it sounds because you don’t have pure sine waves. harmonics from lower octaves will, by design of the piano, interfere with and appear to be higher octaves as well.

    • Dazzam says:

      Another problem could be, that the lead pipes for the individual keys in the pianola may be broken, and you cannot get a complete audio recording.

  5. echodelta says:

    Another wonderful example of “when it comes to music digital was first”. Although 1860 is the first sound recording made thru visual interpolation. Pinned barrels go back half a millennium. There was an experimental device called the melograph. Perhaps there are lines on the back of some papers in libraries that can be converted into music and be attributed to Mozart or Beethoven.

  6. Great hack! It must be good it’s troll worthy.

    And for the Nay sayers:

    Yes but can you write python code using MATLAB? Then we’ll start to be impressed…

  7. strider_mt2k says:

    How about Monty Python Nazis?

  8. Luke says:

    I was expecting something about direct conversion to midi while he was at it. That sounds straightforward from what was done.

    Overall, this is a neat thing. Well done!

  9. Andrew says:

    This reminds me of a project I’ve been thinking about. I have a number of 80 column Fortran IV punch cards from my earliest computing days and I was thinking of making something to read them with a mechanical feed system to move each one past a column of LEDs and light sensors but maybe I should just use my Raspberry Pi with a camera module, take a succession of stills and process the images to create the original Fortran code.

    • chango says:

      Why not both? Mechanical feeder passing each card in front of the camera.

      • Andrew says:

        Yes, I could use a feed system but it would take a lot longer to make it than the time saved for the few hundred cards I have.

        In my original idea with a column of sensors a card transport system is more useful as positioning each card for each column in alignment with the sensor array by hand would be a real pain but with a still photo of each card the ‘scanning’ would be in the software.

  10. Hirudinea says:

    What I really like about this is that he couldn’t find any sheet music for it so he converted a piano roll into sheet music with a great hack.

  11. Frode says:

    Nice work!
    I’m working on a somewhat similar projects, but with scans of perforated metal discs for old music boxes instead of paper tape. The really notable differences with the discs are: more noise, one of the axes are radial and the requirement of accurate center-hole placement. Otherwise it’s much the same.

    Much noise can be dealt with using filters, the radial axis can be dealt with using standard geometry, and the centerhole detection might be automated with a bit of shape analysis.

    • Dazzam says:

      I’d be interested to hear how you go with that. I have a music box disc with a wonderful tune and have not found an arrangement like it anywhere.
      (grr, can’t edit posts. I forgot to tick the ‘notify’ check boxes)

      • Frode says:

        It’s working out rather great. The tool I’ve made have been able to have discs manually analyzed for some time (by marking every single needle by clicking at them), but the thing I’m working on which is the goal of the project, is to automatize all of it.

        Depending on the disc (in my case Symphonion nr.4/24cm/9.5″ 72-track discs), doing a rough markup manually takes about 2 hours. It takes an additional 1-2 hours to apply fine ajustements so the bass and bright tones hit exactly right. An automated analysis takes less than 2 minutes.

  12. Boris says:

    how about some Nancarrow?

  13. pwordupdated says:

    Hi there, that was a brilliant video , Can the user/ or player of that piano play this type of music too ?

    > spotify:track:34LYvEzYNeuDOhKnKBO8dl

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 96,699 other followers