Learning new instruments is never a simple task on your own; nothing can beat the instant feedback of a teacher. In our new age of AI, why not have an AI companion complain when you’re off note? This is exactly what [Ada López] put together with their AI-Powered Piano Trainer.
The basics of the piano rely on rather simple boolean actions, either you press a key or not. Obviously, this sets up the piano for many fun projects, such as creative doorbells or helpful AI models. [Ada López] started their AI model with a custom dataset with images of playing specific notes on the piano. These images then get fed into Roboflow and trained using the YOLOv8 model.
Using the piano training has the model run on a laptop and only has a Raspberry Pi for video, and gives instant feedback to the pianist due to the demands of the model. Placing the Pi and an LCD screen for feedback into a simple enclosure allows the easy viewing of how good an AI model thinks you play piano. [Ada López] demos their device by playing Twinkle Twinkle Little Star but there is no reason why other songs couldn’t be added!
While there are simpler piano trainers out there relying on audio cues, this project presents a great opportunity for a fun project for anyone else wanting to take up the baton. If you want to get a little more from having to do less in the physical space, then this invisible piano is perfect for you!
Bet it can’t crush your fingers the way my piano teacher used to though.
Can we just get realtime MIDI data from video of someone playing piano?
When I read the article, I thought the system was able to track your fingers positions, their state (are they curved correctly or straight), the timing, the prepare for the next note, and so on. In fact, it’s dumber than a simple MIDI keyboard. A dumb keyboard will know when a key is pressed with 100% success, it’ll know the force provided with 100% success too. This will miss keys, will not track anything about your fingers, it’s simply AI boolcheat.
A hybrid perhaps of a Yamaha Disklavier, with maybe some hall/optical switches, and an Intel Realsense for better depth perception. More and better information for the AI.
Stop calling everything AI … thats computer vision
Umbrella term for a host of technologies.
can’t we just call it ‘binfire’ instead?
IMHO, one of many POTENTIALLY useful AIs, but not the most critical. You’d still need a mentor. Not merely teacher, mentor that would see your every mistake made and, ideally, prevent those from happening in the first place. AI-wise I am not sure it can anticipate things that well. Yet. Maybe in the future, once it will learn to tailor to one particular person, quirks and all.
(Spoiler – classical piano background; the learning curve for mastering the piano, or any instrument, really, won’t get any flatter with any new uber technology – once more, “Fahre fort, übe nicht allein die Kunst, sondern dringe auch in ihr Inneres; sie verdient es. Denn nur die Kunst und die Wissenschaft erhöhen den Menschen bis zur Gottheit.” – the loose/romantisized translation is – “Don’t only practice your art, but force your way into its secrets, for it and knowledge can raise men to the divine.” Ludwig van Beethoven).
i really understand the temptation to approach music from this direction, as a techy, but it isn’t fruitful.
and i think that’s really the thing that bums me out the most about hackaday…just a constant drumbeat “i want to brag about the build of something unusable / unused that i made.” there are a ton of people out there using their hacks, but of course they aren’t content creators so we never hear from them. i guess that’s intrinsic and i should just move on. really, i’m part of the problem.
but anyone who uses a tool like this to try to learn piano is soon going to run into its limits. i know 100% if i had done this, i’d write in my project diary about how unsatisfactory it is. instead of publishing it. (in fact, i did this, using MIDI, 15 years ago and it was so unsatisfactory that i didn’t even bother to write the entry, i just dropped it on the ground — that project’s diary is already full of criticism of my other failed add-ons)