Piano Genie Trained a Neural Net to Play 88-Key Piano with 8 Arcade Buttons

Want to sound great on a Piano using only your coding skills? Enter Piano Genie, the result of a research project from Google AI and DeepMind. You press any of eight buttons while a neural network makes sure the piano plays something cool — compensating in real time for what’s already been played.

Almost anyone new to playing music who sits down at a piano will produce a sound similar to that of a cat chasing a mouse through a tangle of kitchen pots. Who can blame them, given the sea of 88 inexplicable keys sitting before them? But they’ll quickly realize that playing keys in succession in one direction will produce sounds with consistently increasing or decreasing pitch. They’ll also learn that pressing keys for different lengths of times can improve the melody. But there’s still 88 of them and plenty more to learn, such as which keys will sound harmonious when played together.

Piano Genie training architectureWith Pinao Genie, gone are the daunting 88 keys, replaced with a 3D-printed box of eight arcade-style buttons which they made by following this Adafruit tutorial. A neural network maps those eight buttons to something meaningful on the 88-key piano keyboard. Being a neural network, the mapping isn’t a fixed one-to-one or even one-to-many. Instead, it’s trained to play something which should sound good taking into account what was play previously and won`t necessarily be the same each time.

To train it they use data from the approximately 1400 performances of the International Piano e-Competition. The result can be quite good as you can see and hear in the video below. The buttons feed into a computer but the computer plays the result on an actual piano.

For training, the neural network really consists of two networks. One is an encoder, in this case a recurrent neural network (RNN) which takes piano sequences and learns to output a vector. In the diagram, the vector is in the middle and has one element for each of the eight buttons. The second network is the decoder, also an RNN. It’s trained to turn that eight-element vector back into the same music which was fed into the encoder.

Once trained, only the decoder is used. The eight-button keyboard feeds into the vector, and the decoder outputs suitable notes. The fact that they’re RNNs means that rather than learning a fixed one-to-many mapping, the network takes into account what was previously played in order to come up with something which hopefully sounds pleasing. To give the user a little more creative control, they also trained it to realize when the user is playing a rising or falling melody and to output the same. See their paper for how the turned polyphonic sound into monophonic and back again.

If you prefer a different style of music you can train it on a MIDI collection of your own choosing using their open-sourced model. Or you can try it out as is right now through their web interface. I’ll admit, I started out just banging on it, producing the same noise I would get if I just hammered away randomly on a piano. Then I switched to thinking of making melodies and the result started sounding better. So some music background and practice still helps. For the video below, the researcher admits to having already played for a few hours.

This isn’t the first project we’ve covered by these Google researchers. Another was this music synthesizer again using neural networks but this time with a Raspberry Pi. And if our discussion of recurrent neural networks went a bit over your head, check out our overview of neural networks.

Continue reading “Piano Genie Trained a Neural Net to Play 88-Key Piano with 8 Arcade Buttons”

Listen To A Song Made From Custom Nintendo LABO Waveform Cards

[Hunter Irving] has been busy with the Nintendo LABO’s piano for the Nintendo Switch. In particular he’s been very busy creating his own custom waveform cards, which greatly expands the capabilities of the hackable cardboard contraption. If this sounds familiar, it’s because we covered his original method of creating 3D printed waveform cards that are compatible with the piano, but he’s taken his work further since then. Not only has he created new and more complex cards by sampling instruments from Super Nintendo games, he’s even experimented with cards based on vowel sounds in an effort to see just how far things can go. By layering the right vowel sounds just so, he was able to make the (barely identifiable) phrases I-LIKE-YOU, YOU-LIKE-ME, and LET’S-A-GO.

Those three phrases make up the (vaguely recognizable) lyrics of a song he composed using his custom waveform cards for the Nintendo LABO’s piano, appropriately titled I Like You. The song is at the 6:26 mark in the video embedded below, but the whole video is worth a watch to catch up on [Hunter]’s work. The song is also hosted on soundcloud.

Continue reading “Listen To A Song Made From Custom Nintendo LABO Waveform Cards”

Pulling Music Out Of Thin Air with a Raspberry Pi

Pianos are great instruments, but being rather heavy and requiring a fair amount of space they are certainly not known for their convenience. Sure, there are more portable varieties available, but they rarely resemble the elegance and classiness of a grand piano. One option is of course to build a downscaled version yourself — and since you’re already customizing the instrument, why stop at the way you play it. [2fishy] didn’t stop there either and ended up with a wooden, space friendly, light controlled piano housing a Raspberry Pi.

Inspired by the concept of a laser harp, [2fishy] followed the same principle but chose a simpler and safer alternative by using LEDs instead. For each playable tone, a LED is mounted opposite a light dependent resistor, creating an array of switches that is then connected to the Raspberry Pi’s GPIO pins. A Python script is handling the rest, polling the GPIO states and — with a little help from pygame, triggering MIDI playback whenever the light stream is interrupted.

There are enough LED/LDR pairs to play one full octave and have some additional control inputs for menu and octave shifting. This concept will naturally require some adjustments to your playing — you can get an idea of it in the demonstration video after the break. And if this design is still not the right size for you, or if you prefer to play in total darkness, this similar MIDI instrument using ultrasonic distance sensors could be of interest.

Continue reading “Pulling Music Out Of Thin Air with a Raspberry Pi”

Flexiphone Rises from the Ashes of Broken Instruments

The mechanics of an old Rhodes Piano, and a set of chromatic saucer bells rescued from a reed organ. What do these two things have to do with each other? If you’re [Measured Workshop], they are the makings of a new instrument. The Flexiphone is a transposable instrument with a piano keyboard and interchangeable sound source.

The Rhodes is a great stage instrument. Unlike a piano with strings, it uses tines mounted above the key mechanism. It is also relatively compact for an analog instrument. This made it perfect as a donor for the Flexiphone’s keyboard. [Measured Workshop] cut they mechanism down to 30 keys, just under 2 octaves. The key mechanism was also cleaned up and restored with new felt.

The sounding portion of the Flexiphone is a set of chromatic saucer bells. The bells are mounted on a felt covered threaded rod, which itself sits in a wood frame. The bell frame sits on top of the base in one of three slots. Each slot is a halftone transposed from the last. Simply moving the bells allows the player to transpose the entire instrument. The bells and their rod frame can also be completely removed and replaced with any other sound source.

The Flexiphone sounds great — sometimes. As [Measured Workshop] says, bells contain many harmonics. playing single or double notes sounds rather sweet, but chords can sometimes become a shrill assault on the ears. Still, it’s an awesome hack with plenty of potential for future mods.

Continue reading “Flexiphone Rises from the Ashes of Broken Instruments”

Putting the Pi In Piano

Working on a PhD in composition, [Stephen Coyle] spends a fair bit of time at his electric keyboard. Setting himself up to work can be a bit of a task, so he felt he could improve the process and make it easy as Pi.

Finding it an odious task indeed to use notation software, connecting his laptop to his keyboard is a must — avoiding a warren of wires in the move is a similar priority. And, what if he could take advantage of the iPad’s unique offerings too? Well, a Raspberry Pi Zero W running Ravelox — an RTP MIDI protocol — makes  his music available on his network to record on whichever device he pleases.

Continue reading “Putting the Pi In Piano”

The Tiniest Of 555 Pianos

The 555 timer is one of that special club of integrated circuits that has achieved silicon immortality. Despite its advanced age and having had its functionality replicated and superceded in almost every way, it remains in production and is still extremely popular because it’s simply so useful. If you are of A Certain Age a 555 might well have been the first integrated circuit you touched, and in turn there is a very good chance that your project with it would have been a simple electric organ.

If you’d like to relive that project, perhaps [Alexander Ryzhkov] has the answer with his 555 piano. It’s an entry in our coin cell challenge, and thus uses a CMOS low voltage 555 rather than the power-hungry original, but it’s every bit the classic 555 oscillator with a switchable resistor ladder you know and love.

Physically the piano is a tiny PCB with surface-mount components and physical buttons rather than the stylus organs of yore, but as you can see in the video below the break it remains playable. We said it was tiny, but some might also use tinny.

Continue reading “The Tiniest Of 555 Pianos”

Pumpkin Piano Promises a Gourd Time

Fall – it’s that time of year that brings falling leaves, Hallowe’en, and a pumpkin version of everything that you hold dear. In this case, it’s not a latte – it’s [Robert Vorthman]’s Pumpkin Piano.

[Robert] took a straightforward approach to the build, pressing a Raspberry Pi into service as the backbone of the operation. This is combined with an Adafruit breakout board for the MPR121, which is a chip that provides 12 capacitive touch-sensitive inputs. These are connected to the bountiful produce which make up the piano keys in this fun holiday hack. [Robert] uses some Python code that talks to fluidsynth, a software synthesizer that uses Soundfont files to create different sounds. It’s all wrapped up with some Neopixels that flash when each vegetable is triggered.

The build would make a great party piece for just about any fall gathering, and [Robert] has done a great job of rolling up all the hardware and software required in the write-up. For another take on a vegetable-based orchestra, check out last year’s Harpsi-gourd.