Facebook’s Universal Music Translator

Star Trek has its universal language translator and now researchers from Facebook Artificial Intelligence Research (FAIR) has developed a universal music translator. Much of it is based on Google’s WaveNet, a version of which was also used in the recently announced Google Duplex AI.

Universal music translator architectureThe inspiration for it came from the human ability to hear music played by any instrument and to then be able to whistle or hum it, thereby translating it from one instrument to another. This is something computers have had trouble doing well, until now. The researchers fed their translator a string quartet playing Haydn and had it translate the music to a chorus and orchestra singing and playing in the style of Bach. They’ve even fed it someone whistling the theme from Indiana Jones and had it translate the tune to a symphony in the style of Mozart.

Shown here is the architecture of their network. Note that all the different music is fed into the same encoder network but each instrument which that music can be translated into has its own decoder network. It was implemented in PyTorch and trained using eight Tesla V100 GPUs over a total of six days. Efforts were made during training to ensure that the encoder extracted high-level semantic features from the music fed into it rather than just memorizing the music. More details can be found in their paper.

So if you want to hear how an electric guitar played in the style of Metallica might have been translated to the piano by Beethoven then listen to the samples in the video below.

Continue reading “Facebook’s Universal Music Translator”

Synthesizing Strings On A Cyclone V

Cornell students [Erissa Irani], [Albert Xu], and [Sophia Yan] built a FPGA wave equation music synth as the final project for [Bruce Land]’s ECE 5760 class.

The team used the Kaplus-Strong string synthesis method to design a trio of four-stringed instruments to be played by the Cyclone V FPGA. A C program running on the development board’s ARM 9 HPS serves as music sequencer, controlling tempo and telling the FPGA which note to play.

The students created versions of four songs, including “Colors of the Wind” from the Pocahantas soundtrack, “Far Above Cayuga’s Waters” (Cornell’s alma mater) and John Legend’s “All of Me”. A simple GUI allows the viewer to select a song and to choose which instrument or instruments to play, providing multiple variations for each song.

Thanks, [Bruce]!

Continue reading “Synthesizing Strings On A Cyclone V”

The 35 Year Music Synthesizer That Spawned Chiptune

If you are a certain age, MOS6581 either means nothing to you, or it is a track from Carbon Based Lifeforms. However, if you were a Commodore computer fan 35 years ago, it was a MOS Technologies SID (Sound Interface Device). Think of it as a sound “card” for the computers of the day. Some would say that the chip — the power behind the Commodore 64’s sound system — was the sound card of its day. Compared to its contemporaries it had more in common with high-end electronic keyboards.

The Conversation has a great write up about how the chip was different, how it came to be, the bug in the silicon that allowed it to generate an extra voice, and how it spawned the chiptune genre of music. The post might not be as technical as we’d do here at Hackaday, but it does have oscilloscope videos (see below) and a good discussion of what it took to create music on the device.

Continue reading “The 35 Year Music Synthesizer That Spawned Chiptune”