ChatGPT is being asked to handle all kinds of weird tasks, from determining whether written text was created by an AI, to answering homework questions, and much more. It’s good at some of these tasks, and absolutely incapable of others. [Filipe dos Santos Branco] and [Edward Gu] had an out of the box idea, though. What if ChatGPT could do something musical?
They built a system that, at the press of a button, would query ChatGPT for a 10-note melody in a given musical key. Once the note sequence is generated by the large language model, it’s played out by a PWM-based synthesizer running on a Raspberry Pi Pico.
Ultimately, ChatGPT is no musical genius. It’s simply picking a bunch of notes from a list that are known to work together melodically; that’s the whole point of musical keys. It would have been wild if it generated some riffs on the level of Stairway to Heaven or Spontaneous Devotion, but that might be asking for too much.
Here’s the question, though. If you trained a large language model, but got it to digest sheet music instead of written texts… could it learn to write music in various genres and styles? If someone isn’t working on that already, there’s surely an entire PhD you could get out of that idea alone. We should talk!
In any case, it’s one of the more creative projects from the ever-popular ECE 4760 class at Cornell. We’ve featured a bunch of projects from the class over the years, and noted how the course now runs on the RP2040.
API key right in the thumbnail. Fun.
It’s also in the code provided at the link. Way easier to copy and paste lol.
artificial…not artistic intelligence
Me: If I upload sheet music, can you analyze it?
ChatGPT: Yes, I can analyze sheet music if you upload an image of it. I can help with identifying notes, rhythms, key signatures, and provide some insights into the structure and style of the piece. Please upload the sheet music you’d like me to analyze.
Me: Can you generate new compositions based on the sheet music you analyze?
ChatGPT: Yes, I can generate new compositions based on the sheet music you provide. After analyzing the sheet music, I can create variations, develop new themes inspired by it, or compose a piece in a similar style. Feel free to upload the sheet music you’d like to use as a basis for a new composition.
Will be interesting to try out! This is a much more advanced answer than what I saw 6 months ago.
Oh that’s too funny. It’s very clear in the video as well. Skip to 1:52.
I’m interested to see someone merge the analog synth with fine-tuning like we see in this tutorial: https://huggingface.co/blog/juancopi81/using-hugging-face-to-train-a-gpt-2-model-for-musi
This and the midi keyboard synth a few days ago are all available on the instructor (Hunter Adams) YouTube channel. Probably worth anybody interested in seeing a ton of fpga and rp2040 projects to just browse and watch what they’re interested in. Really cool to see the creativity and variety en masse from this great class. Hunter also has great tutorial lectures on using rp2040, and pios. https://youtube.com/@hunteradams9430?si=2j6jLRKuASjh8ett
This is how you parse MIDI on an ARM cpu.
Written for the LPC810/812 but easy ported to anything.
https://gist.github.com/piedoom/2635ef7589e1589c32500e13a0b89a48
I have heard that they are using stable diffusion but training it on spectrograph images of songs.
“Would you like to play a game? How about a nice game of chess?”
I haven’t done a PhD on AI generated music, but I have explored it somewhat. E.g. Here are 4 songs made with ai, and 1 that’s not. If you can’t tell the difference, does it matter?
https://nodemusic.bandcamp.com/album/co-written
@Lewin Day: Can you remove the title image?
The Video is down because of the API key and they removed it from the site…
Things in this space are moving pretty quickly…
https://youtu.be/b72DrL-qito
Been basically doing it with modular for years.
Hello,
ChatGPT wrote and composed a full-length album of Hardcore Techno for me.
It’s song- and track-writing skills are simply exceptional.
And I documented every part of it, so that others could do it, too :-)
https://doomcorerecords.bandcamp.com/album/the-cosmic-longing-of-an-artificial-soul
https://laibyrinth.blogspot.com/p/how-to-create-music-with-chatgpt.html
https://lowentropyproducer.blogspot.com/2024/01/i-wrote-whole-techno-album-together.html
Yours Truly,
Sönke / Low Entropy