[Urpo Lankinen] has a father who is a huge movie buff, and once you care deeply about something it begins appearing everywhere you look. While driving on a back-country road one day, [Urpo]’s dad noticed that the shadows of the trees on the road looked like an optical film soundtrack, so it was up to the son to make it happen.
Early sound-on-film technologies like the RCA Photophone, and Movietone recorded audio onto the film stock with a variable area exposure. This exposure corresponds to the waveform of the audio signal. [Urpo] figured that his small digital video camera served the same purpose as the audio sensors inside the projector, so he put a piece of tissue paper over the lens and wound up with a video that was just frames of gray.
[Urpo] built an app in Processing that averages the pixels in each frame of the video. Of course, recording at 30 frames/second won’t produce any audio this way, so he modulated a triangle wave with this data in Audacity. In the end it really doesn’t sound like much, but it’s great to see such a geeky build.
We’d love to post the video after a break but [Urpo] doesn’t believe in YouTube embeds. We’ll honor his wishes, so you can check out the video here.
modulating the volume doesn’t make much sense. why not quantize to discrete musical notes and modulate the pitch?
I agree, audio on old film tracks aren’t modulated tones! The ARE the audio waveforms!
Actually it does make sense, but the creator of the video still got the execution wrong.
The optical audio track on a film strip (not the digital ones) does record the amplitude. The width of the white area for any given horizontal slice across the film is the amplitude (volume) at that given time.
That’s how digital audio works too, btw. You record the amplitude of the audio signal during a given sample. If you record 44100 samples per second, with 16bit resolution for the amplitude you get CD quality sound.
There’s no such thing as frequency/pitch at an instant of time.
So the concept of light intensity to audio amplitude is correct. However if he averaged the intensity of light per frame (1 frame = 1 sample) he’d need to speed up his video anywhere from 500 to 1500 times to have it sound like something in a frequency high enough to hear.
Modulating it with another wave makes no sense at all in the analogy from optical film scores.
I think someone could easily do this and it might sound pretty cool even (especially because the trees on the side of the road are in somewhat regular intervals… it wouldn’t be just random noise, you’d hear something)
Doesn’t believe in embeds? Not interested in watching then.
I thought I read somewhere about putting music into highways by altering the distance between the ‘Wake up’ groves in the sides of the roads that change the frequency your tires make as you run over them. Thought that was pretty cool.
http://www.mymodernmet.com/profiles/blogs/honda-music-road-highway-to
Good try, I find your Dad’s idea very poetic, but a video camera is the wrong sensor.
Using 30fps video means the highest possible frequency is 30 Hz. Human hearing range is between 20 Hz and 20 hHz (Wolfram Alpha).
To replicate how analog optical tracks work, you’d just need to take the signal from a photoresistor (or a phototransistor), amplify it and feed it directly to a speaker. That’s how it works on sound-on-film. The technology dates back to the 1920s (Wikipedia), what do you expect? ;+)
I’d be very curious to hear what kind of sound you get.
sorry had, but this isn’t very good at all
covering with toilet paper defeats the purpose. the point of audio tracks stored like that is that the pointy bits are the sound, no averaging involved.
He doesn’t believe in embeds? like he thinks they don’t exist?
There is a stretch of road in Oita prefecture in Japan with a musical road surface. I think they were using different forms of asphalt to make the different notes, rather than having grooves cut in the surface.
It’s videos like this that make me wish YouTube had a ‘Play faster’ button.
Here is a link to the output:
http://www.youtube.com/watch?v=nIWdHjIQkL4#t=213s
Seems to me that the diffuser is the wrong approach. He’s basically putting a big triangular moving average on top of the wave form.
He should just pick a row of pixels from the video, trim them to the width of the road surface, run a threshold filter, then take the number of black pixels as the sample. Repeat for each frame.
Might be able to increase the sample rate by figuring how many pixels the shadows move between frames, then pull that many rows, so you get parallel sampling.
Oops, it embeds youtube videos; I meant:
watch?v=nIWdHjIQkL4#t=213s
Also, didn’t mean to sound so negative. This is a cool idea, and the video was nicely assembled (I’d just already read the description here, so didn’t want to read cards in a video).
It would be interesting to see how different sampling methods produce different sounds.
wow that’s not hyped up at all
To actually duplicate how the sound tracks work, would it be best to send raw video output to an audio amplifier somehow? Perhaps have the video illuminate a photo resistor? A lot of work could be eliminate by having the photo resistor “read” the road directly The result could be a boring as thump…thump…thump..thump… thump…thump; or as scary a a satanic message :). While it may be minor point, but toilet paper was used not tissue paper. there is a difference.
Hmm. Sounds like a sine wave linked to a photoresistor. Dude could have done so much more after getting this far.
http://www.youtube.com/watch?v=ssJutXkpSlY
Note the video is synced to the images.
^^
I’ve never been a fan of the Chemical Brothers, but that’s f ck ng awesome.
Doesn’t believe in embeds. Too much fluff in the video clip, with results only presented at the end. Modulation is done wrong. Interesting idea, but poorly presented IMO.
the hell… this is a sound track of what???
just sounds like a bunch of ooohhhhh,blalbal,oooooooooooooooo.
That is not even a song. This sucks.
In the words of the great Carlin: “I’d rather watch two flies fuck”
cool idea, but like mentioned in the article it “really doens’t sound like much”. seems too complicated of a setup just to modulate the triangle. gave me some good ideas though. the TSL230R might be interesting here.
Why not average every line of every frame and use that to generate the wave form? Of course looking at that one frame the result would be a good number of Sawtooth waves ending in a really big sawtooth from the sky and the bottom of the next frame but that would be if you set the origan at the bottom.
http://ericarcher.net/devices/sound-cameras/
I think something like that makes more sense than 20-30 samples/second used AM a tone
That was a pretty pompous video, especially in light of the frankly underwhelming outcome.
Also, the choice of using shades of grey to modulate a triangle wave seemed completely arbitrary. That’s certainly not how an optical soundtrack works, so I wonder what the point is…
Sorry guys, it’s not like I get a kick out of putting down people’s work, but this time I’m seriously unimpressed. If the video had been just a bit less pretentious, though, I’d have kept my objections to myself, as I usually do in these cases.