With his latest project, [Roni Bandini] has simultaneously given the world a new type of audiobook and music. Traditional audiobooks are basically the adult equivalent of having somebody read you a bedtime story, but BookSound actually turns the written word into electronic music. You won’t be able to boast to your friends that as a matter of fact, you have read that popular new novel, but at least you might be able to dance to it.
[Roni] says he’s still working on perfecting the word to music mapping, so the results shown in the video after the break are still a bit rough. But even in these early stages there’s no denying this is an exceptionally unique project, and we’re excited to see where it goes from here.
Inside the classy looking 3D printed enclosure is a Raspberry Pi, an OLED display, and the button and switch which make up the extent of the device’s controls. At the end of the arm is a standard Raspberry Pi Camera module, which gives the BookSound a bird’s eye view of the book to be songified.
To turn your favorite book into electronic beats, simply open it up, put it under the gaze of BookSound, and press the button on the front. Because the Raspberry Pi isn’t exactly a powerhouse, it takes about two minutes for it to scan the page, perform optical character recognition (OCR), and compose the track before you start to hear anything.
If you’re wondering what the secret sauce is to turn words into music, [Roni] isn’t ready to share his source code just yet. But he was able to give us a few high-level explanations of what’s going on inside BookSound. For example, to generate the song’s BPM, the software will count how many words per paragraph are on the page: so a book with shorter paragraphs will consequently have a faster tempo to match the speed at which the author is moving through ideas. Similarly, drum kicks are generated based on the number of syllables in each paragraph. In the future, he’s looking at adding “lyrics” by running commonly used words on the page through a text to speech engine and inserting them into the beat.
We’ve seen practical applications of OCR on the Raspberry Pi in the past and even similar looking book scanning arrangements. But nothing quite like BookSound before, which at this point, is really saying something.
Great project, great write up
Thanks, Prof.
Hmm. Not bad! I could jam to that, if I could play an instrument. Or, you could add some synth deedley-doos and some samples from old sci-fi movies and call it techno.
Good idea. Thanks, Eric.
Love this idea. Wrote a small program in college that turned any file into “music” (more like noise) – this is way better and something way more thought out. To the author: are you using C-Sound or CLAM?
Thanks, Casey. No, I’m not using CSound or CLAM.
This could be a brilliant application for Deep learning (AI). Could feed it lyrics and music of known songs and let it learn how to change the words to music.
Thanks for thr suggestion, Setvir.
Making hard style from your book lol ;)
:)
Very cool project!
Thanks, Matt Daemon. Great nick, BTW.
Ha, that’s pretty neat. I’m looking forward to the lyrics part–I was half expecting some old apple voice to start reading the book to a tune with the beat behind it.
Hi TGT, In the current version some TTS words extracted form the page are used in the track.
I have just uploaded a new demo with BookSound software improvements https://www.youtube.com/watch?v=A9x9hl_eTYE