Learning to read sheet music is a challenge for new musicians, so a group of Cornell students decided to make a robot do it instead. For their final project, they built a robotic sheet music reader (link warning: this page seems to automatically start a .mov file download when loaded).
As an input, the robot takes a piano roll. This is a long strip of paper with music printed on it, which can easily be fed through the reader. A rotational servo keeps the strip feeding at a constant speed, and passes it through the reader.
The reader is based on a Melexis MLX75306 linear optical array. This IC is a 142 x 1 array of photodiodes, which is designed for fuel quality sensing in cars. In this project, it’s been repurposed as a camera to read the music as it passes by. An array of LEDs illuminates the piano roll, providing a more accurate reading.
The components are connected to an Atmel ATmega1284P, which does all the required control and processing. It creates a MIDI output of the piano roll, which can be connected to any hardware or software synthesizer.
Interesting hack but how this relates to sheet music is beyond me, the “piano roll” seems to be a proprietary format with little or no relation to sheet music. A scanner that would actually read sheet music would be quite an accomplishment, this device, not so much.
I tend to agree with Miles. In effect this is an overly complex paper tape reader.
OCR for “tadpoles” doesn’t sound like it is beyond the scope of current technology… and indeed a quick Google provides a number of such applications.. http://www.synthesiagame.com/wiki/Converting_Sheet_Music
I agree with both of you and thought of changing g this during editing.
But figuring out what to call it instead proved difficult. This is a concise way to get the meaning of the machine across. Plus, I think it is a proof of concept for a more complex machine that could scan each stave of traditional printed music in a similar way.
Piano Roll To MIDI Converter. This is what it is. No sheet music here. Yes piano roll here.
Yea, but it’s not “sexy” how about “Playing a 19th century iPod”?
I’m not sure about the truth behind this, but when I was a kid, in 1984 (or thereabouts) Waseda University in Japan came up with this robot, the WABOT-2:
http://www.humanoid.waseda.ac.jp/booklet/photo/WABOT-2-1984.jpg
Supposedly, it could play the synthesizer by reading normal sheet music (along with several other “feats”, including improvising a tune to match a singer, and singing itself). It used a large number of processors to perform these tasks, and was considered one of the most advanced robots in the world at the time.
Surely – if it is true that this robot in 1984 could read sheet music and play it – that certainly today we could have a scanner (or camera) read in sheet music and convert it to MIDI. It would certainly seem to be a greater possibility today, than it would have been back in 1984.
Well – I found this:
http://rraj.rsj-web.org/en_atcl/224
At the bottom are some links to research papers regarding the WABOT-2 – detailing construction, operation, and how the vision system works for reading sheet music. They’re probably pretty interesting, if you’re fluent in the language.
I find it strange in those research papers I linked to, that they are written in japanese, but that the abstract, plus all of the diagrams (among others) are written in english; why wouldn’t these have been all written in english for an international audience? Why would only the diagrams, etc be in english? If they were aiming these papers toward a mainly japanese audience, why wouldn’t the diagrams be in japanese as well? I’m not complaining, I’m just perplexed by it…
Maybe because the figures in these Japanese papers are derived from English written papers?
If You Google for “wabot2 posture of hand” You will find English written papers.
Rollscanning is major project worldwide to preserve the heritage of the first generation of digital music. Hundreds of thousands of recordings made from 1890’s to the 1930’s are on acid bearing paper now in poor condition. Perfect playback is possible up until the paper crumbles. By scanning with light instead of air the paper can safely be read, damage can be edited out.
A lot of these records are NOT in the Library of Congress or other bastions of preservation, their producers are long gone. They are in closets and basements and piano room cabinets.
A copyrighted song was typically available in three formats. Sheet music, piano roll, and phonograph record.
Hi , The idea looks good to me eventhough many others have opined otherwise. I am particularly interested in the line scan sensor you have used. By the way have you used any lens?