It’s easy to dismiss radio as little more than background noise while we drive. At worst you might even think it’s just another method for advertisers to peddle their wares. But in reality it’s a snapshot of the culture of a particular time and place; a record of what was in the news, what music was popular, what the weather was like, basically what life was like. If it was important enough to be worth the expense and complexity of broadcasting it on the radio, it’s probably worth keeping for future reference.
But radio is fleeting, a 24/7 stream of content that’s never exactly the same twice. Yet while we obsessively document music and video, nobody’s bothering to record radio. You can easily hop online and watch a TV show that originally aired 50 years ago, but good luck finding a recording of what your local radio station was broadcasting last week. All that information, that rich tapestry of life, is gone and there’s nothing we can do about it.
Or can we? At HOPE XII, Thomas Witherspoon gave a talk called “Creating a Radio Time Machine: Software-Defined Radios and Time-Shifted Recordings”, an overview of the work he’s been doing recording and cataloging the broadcast radio spectrum. He demonstrated how anyone can use low cost SDR hardware to record, and later play back, whole chunks of the AM and shortwave bands. Rather than an audio file containing a single radio station, the method he describes allows you to interactively tune in to different stations and explore the airwaves as if it were live.
Modern Take on a Classic Technique
You might think that such radio trickery is a product of modern hardware and software, but in fact the methods Thomas and his group of radio archivists use have considerably more retro beginnings. As far back as the 1980’s DXers, radio hobbyists that look specifically for distant signals, found that if they connected the intermediate frequency (IF) output of their radio to a VCR they could capture whatever their antenna was picking up for later analysis. When the tape was played back through the antenna port of the radio, they could tune to individual frequencies and search for hard to hear signals.
Of course the utility of this method wasn’t limited to just weak signals. It allowed radio operators to do things that would otherwise be impossible, like going back and listening to different news broadcasts that were aired at the same time. A few DXers realized there was a potential historical value to such recordings, and some of these early tapes were saved and wound up becoming part of the collection Thomas has been building and offering up as a podcast.
The modern version of this technique replaces the AM or shortwave receiver with any one of a number of affordable SDR devices, and the VCR has become a piece of software that can dump the SDR’s output to a file. This file can then be loaded up in a compatible SDR interface program, such as HDSDR, in place of an actual radio.
Storing History
Thomas envisions a future where researchers will be able to sit down at a kiosk and browse through the radio broadcasts from a given time and place, the same way a microfilm machine is used to look at a newspaper from decades past. But while making these recordings is now cheaper and easier than ever before, there are still logistical issues that need to be solved before that can happen. Chief among them: how do you store it all?
Thomas mentions that a single day’s recording of the AM broadcast band will result in roughly 1 TB of data. Potentially some compression scheme could be developed which would scan the recordings to isolate the viable signals and delete the rest. Another approach would be a sort of ring buffer arrangement, where the system only retains the last few days of recordings unless the user commits them to long-term storage. If something deemed worthy of future study occurs, the ring buffer could be moved to permanent storage so the event as well as the preceding time could be preserved for historical purposes.
Until then, Thomas and his team will keep on recording during noteworthy events. As an example, they made extensive spectrum recordings during the 2016 US Presidential elections, believing it will be a moment future generations will likely want to have as much information on as possible.
Well, considering that a lot of broadcast radio is pre-recorded music, Station Logs, can eliminate a lot of data right there.
And then, a lot of commercials are repeated, so only one of those should needs to be kept along with the station logs.
And if a station airs network news, well, that is probably archived somewhere…
So, then that leaves Talk Radio and Sports Talk Radio…
Noise gate + MP3/AAC style audio compression? I bet you’d knock the bitrate right down.
I think they’re recording more of the spectrum than just one channel…so that kind of trick is not going to fly.
The signals aren’t decoded down to audio yet – which would itself cut the bit rate quite a lot…the idea I’d think is that modulation type, channel occupation, spacing, presence, doesn’t even matter till playback time.
Or that’s what I got without watching the video, just reading the text.
Looking at a waterfall plot, you’ve really only got two dimensions of data: intensity across frequency and time. And there’s tons of correlation across frequency, especially in the empty bands, and there’s a lot known about compressing audio signals over time. There should be plenty of room for clever compression.
Or put another way: the FFT is like tuning into many many (AM) stations at once. You can compress the audio from each, and disregard the ones that are just quiet noise. Extending this to FM is a tad trickier, probably. But you get to ignore all the empty space once you’ve found out that it’s empty.
You _would_ miss spread-spectrum action. But this is just for broadcast radio, right?
What am I missing?
Might be worth looking into FBMC (filter bank multi carrier) techniques. This allows you to break up a single wide-band channel into multiple channels of varying bandwidths / modulations simultaneously using polyphase filter structures. Seems like a good fit for a post-processing technique prior to compression. Obviously you would need FPGA / ASIC to do this real-time.
You’re missing the fact that the whole way lossy audio compression works is from the codec having an idea what types of sounds are noticeable to the human ear. Something that’s really just a great big chunk of spectrum does not have the same characteristics of which parts are important and which not. An audio codec does not know how a particular modulation/encoding scheme works. Especially if you consider that some of these radio channels could be digital.
Now you could do the station-recovery before storage and yeah, just store them each as audio. But that’s not quite such a mind-blowingly flexible proposition.
(TL/DR: the waterfall display isn’t what is recorded – raw time-domain samples are. No complex processing is needed for the recording side. All of the FFT and other fancy stuff is done on playback.)
Elliot: yes, you only have two dimensions of data represented in a waterfall plot, but this is just one way of viewing the data. In reality, 1) that waterfall display doesn’t contain enough information to reconstruct the original signal (it COULD, but this would require a lot more resolution in all three dimensions), and 2) you really have just a one-dimensional array – the time domain samples themselves. I don’t know how, specifically, the recording in the video was made, but in principle it’s pretty “easy”: you bandpass filter the band of interest (to avoid all kinds of aliases being captured), convert it through a mixer to the band a VCR could record, low-pass filter that to remove the sum components, then record it. If they were recording a whole band at a time from I.F. signals in a radio, these would have to be at a high enough I.F. frequency that the whole band of interest was within the passband of the mixer output, like maybe a 5 or 10 MHz I.F, which were both common in both amateur and communications receivers. However it’s done, you have to have at least as much bandwidth in your recording as the width of the band you’re recording. In the case of the U.S. broadcast band, the frequency range is 540-1600 kHz, so you need a band pass of 1060 kHz. From what Wikipedia says, the audio carrier was at 1.6 MHz, so … maybe that was doable without too much pain. It could not have been recording as video, since the video section of a VCR required proper sync pulses to lock the head motor to, which would then put 60 Hz garbage on the whole band, but HiFi VCRs were able to record continuous audio without much crosstalk from the video signal. It would probably help to record only a “black burst” or genlock signal on the video track.
But given that this was somehow done, my point is, what was being recorded was just raw samples of a 1060 kHz band of signals. It’s only by playing this back through SDR software that we see it as a waterfall display. Being a dumb recording, it does not discriminate among the signals within that band; this is done only when we “tune” to them on playback, which is what that SDR software is essentially doing. In fact, if the signal being played back was converted back to its original frequency, then you could connect the output of the player to the antenna terminal of an AM radio and literally tune through the stations as if it were live. Think about that experience for a second – tuning a 1980s AM radio and hearing 1980s radio stations! I think it might be enough to put me over the edge, if I didn’t suspect what was going on.
Yes, this is a terribly inefficient way of recording the AM broadcast band for sure, since it completely wastes the portions of the bandwidth that contain no useful information. If you were interested in just recording what was being broadcast by licensed transmitters, then you could save some bandwidth by using DSP to separate the band into discrete channels, mixing each down to baseband audio, and recording that. An additional benefit of this is that you only need half the bandwidth of each station, since an AM station in the U.S. with a 5 kHz audio bandwidth requires 10 kHz of RF bandwidth, due to being double-sideband modulation. (Ignoring the fact that compatible AM stereo broadcasts in the U.S. modulated the stereo channels on opposite sidebands, but who cares about stereo AM?) This means that you essentially demodulate each station separately, and once you’ve done that, you might as well just delete the channels whose amplitudes are too low to contain any recoverable signal. And THEN, of course, if all you’re interested in are legal transmissions, i.e., licensed AM modulated audio stations, you could compress each channel using the standard modern lossy techniques. The only real advantage of the full-band recording example shown here, is that it would have been very expensive in the 1980s to do this, especially since the lossy compression part hadn’t been invented yet. I think. And of course, it also works for transmissions that were NOT in the strict 10 kHz channels (in the U.S.; 9 kHz in other parts of the world), and it works for other modulation modes as well. In short, it records data you wouldn’t have known would be useful when you recorded it.
I haven’t tried this, but I expect that mp3 compression of a 5 kHz limited audio signal could achieve pretty close to broadcast quality with about 20 kb/s of data. If this is correct, and every channel of the broadcast band had a station present, recording the band this way would require 106 * 20 = 2120 kb/s, or 265 kB/s. There being 86,400 seconds in a day, that’s 23 GB per day, as opposed to the reported 1 TB/day for raw data stated in the article. And again, this assumes that all 106 AM channels are recorded; this would get way better in most markets, where there are relatively few AM stations.
BUT. If hams were using techniques like this, it wasn’t to preserve moments of AM broadcast for posterity – that I’m sure was just a demonstration. Doing a similar trick with any of the amateur bands would be a harder case to compress, since the signals aren’t all the same bandwidth, and certainly not on a regular interval of carrier frequencies.
Other than the “I’ll record the entire band so that someone in the future can find things I don’t even know to look for” factor, it’s pretty wasteful. The saving grace is that this didn’t have to be recorded digitally, since it took advantage of an existing technology that could record that much bandwidth as an analog signal. So if you wanted to have a record of everything broadcast (in the AM band) for the past 24 hours, you could set up 12 VCRs to each record a two-hour segment of the day (I’m assuming you want better s/n than you get from the LP or SLP modes), which even in the 1980s would have been a project within reach for many. Or you could hack a standard VCR cartridge to use as an adapter to take tape from external reels, extending its recording time to any arbitrary length you wanted. Kind of the opposite of a VHS-C adapter.
dcfusor2015: I agree – this would be a complete waste, if it was just recording individual channels. The trick is that it records an entire BAND of frequencies, like the whole AM broadcast band, as a single signal stream.
I know, I used to do it for an entity.
One thing Elliot missed would be things like some kinds of spread spectrum that don’t show above the noise in a simple FFT kind of analysis at all, but later on if you know a key, you can still get them back. Not germane to the stated reason here, but….it was to that entity.
I’m pretty sure various spy agencies have been doing something like this for decades.
They have, at least in bursts. When I was there (late ’70s) work was underway to get essentially DC-1Ghz digitized and written down…it seemed pretty hopeless then with the existing tech, though. The effort, IIRC, did help move A/D design forward, even things like Successive Approximation By Residual Expansion (SABRE) which can be pipelined (coax being the delay line). But even they didn’t seem to have the BW or capacity of memory media – the days of 9 track tape and just beginning of winchester drives – slow and small, even with a big room full. Took a pretty big box on a PDP 11/70 (I know, not the king of the hill, but nice) to even do a real time digital phase locked loop on a 10 mhz IF at the time. Lots of big wire-wrap boards….and only the reduced data could be written down, not the whole IF BW.
I think even today, if you could (and you’d have to do it in a lot of places since signals don’t always propagate to your location) – how in heck would you categorize and search such data? That’s way past what one human can keep track of, even with various help from machines. And if you have to split up the job, then some correlations are going to be missed.
Even Utah can only hold so many bits per person on earth with existing tech now. Friend of mine worked it out assuming the whole building has no air, just SSD flash chips and while it’s a lot, it’s not a lot on that scale at all.
More like one music (or pron) collection/person.
DNA as a storage medium.
Well yes true, trouble is it’s far more mobile and fragile in some respects than most have ever considered, from viral particles in the air we breathe all over the world to phage to bacteria to guts to gonads and so on it goes ad infinitum. One huge mix of potentials for immense permutations eating away or adding or whatever to all life’s genetics in layers upon layers of chaotic fashion. So having another source to add to the mix will make our planetoid Grex one seriously complex new organism. Hey, who knows this might be one esoteric means for a far more subtle alien type life to procreate…
I can’t find any reference to this SABRE. Link?
“It’s easy to dismiss radio as little more than background noise while we drive. At worst you might even think it’s just another method for advertisers to peddle their wares. But in reality it’s a snapshot of the culture of a particular time and place; a record of what was in the news, what music was popular, what the weather was like, basically what life was like. If it was important enough to be worth the expense and complexity of broadcasting it on the radio, it’s probably worth keeping for future reference.”
Certainly good for nostalgia’s sake. For when you can’t sit around listening to some Amos and Andy.
“…if they connected the intermediate frequency (IF) output of their radio to a VCR they could capture whatever their antenna was picking up for later analysis. When the tape was played back through the antenna port of the radio…”
They must have had radios that could tune their own if frequencies then. You didn’t used to see that so much because it required a different band with a different IF. Maybe they played it back into a second radio, probably something meant for listening to VLF signals.
Maybe bad phrasing. A lot of multiple conversion pro superhet receivers had a lot of IF bandwidth at the first IF frequency (often centered around 10 mHz) and it would have quite a few signals in it. Another conversion stage with an appropriate following bandwidth would let you tune to just one of the signals (you hoped).
Analog tape at really fast inches/second could record that 10 mhz IF, then you could play it back _into the IF strip_ and rest of the receiver (not the antenna on the front end).
You ran out of tape real fast – VCRs were a big step forward if you could handle the glitches per rev of the head wheel.
Ah, ok. I was picturing a classic 455khz IF.
And yes, a lot of communications receivers can receive their own intermediate frequencies. Usually at least double-conversion receivers, though.
dcfusor2015: but this was how HiFi VCRs did their audio: they used a 1.6 MHz carrier for the audio, which for some reason was a piece of spectrum not already full of luminance or chroma information, and recorded it on the helical track, superimposed on the video. Separate heads were used for this, and there was enough overlap between the audio heads that the audio signal was continuous, i.e., mostly glitch-free. And it somehow worked – no lie. I don’t KNOW this, but I’m guessing that people were able to hack the audio section of these decks to pass a lot more than the normal audio bandwidth to the “audio” track this way, and if the video signal was just clean black, there wouldn’t be (much) interference between the two.
Yep….
Hmm, received two notifications for this thread and most interesting but, don’t recall seeing this before let alone posting. As it happens it has relevance in near future, maybe hackaday’s binary gestalt is turning prescient – and sooner than expected ;-)
I was hoping you were talking about listening for signals bounced off of far away extrasolar objects, returning to Earth decades later or something like that. Of course it would be pretty far-fetched but I suppose that wouldn’t stop someone from trying and maybe this was an article about some crackpot I mean experimenter trying it.
Maybe I’m just still jaded from reading about pyramid energy in Popular Electronics so many years ago.
1. Develop faster than light travel.
2. Travel the distance (time frame) you want to listen.
3. Tune radio to station. ;)
I think that was covered in a cartoon or something in one of the hobby electronic magazines in the sixties.
Michael
#MeToo
Sometimes there nothing wrong with realizing that moment of life evaporate… get lost forever… never to be heard again… etc. There is beauty in knowing that something is just a single unrepeatable event. Radio is already recorded, just not every station and just not every minute and I’m pretty sure there are good reasons for it (radio stations only airing music, playing the same song once every hour, etc). Just because we can record something, doesn’t mean we should.
But from a technical point of view, I can see the beauty of the project. It is a nice challenge.
Something to think about. Recording radio as a modern-day “Hindenburg” happened, events of 9/11.
Please explain further. I don’t see the connection to 9/11.
I think he meant “historical events”, as in the Hindenburg crash. That particular day was quite historical. It might be interesting to have that day’s broadcasts all stored.
Then again, most of it would just be “WTF? OMG! … … WAR!”
It’s amazing what can trigger nostalgia as you get old.
And of course…
It’s profitable what can trigger nostalgia as others get old.
OMG Look at the case with those glass pop bottles in it in the background of this old photo. That’s a Pepsi Free case! (yes, I recently experienced this one)
I never understood why Pepsi decided to add lemon flavor to the “clear/crystal/whatever” version they marketed decades ago… If removing the carmelized sugar altered the taste of Pepsi, why didn’t they come right out and admit it? And I sort of wondered why it made a difference with people when so many just drink straight from a can and never even see if it is clear or brown.
And why did they keep telling me I had to pay for Pepsi Free?
B^)
Underrated comment
“Sometimes there nothing wrong with realizing that moment of life evaporate… get lost forever… never to be heard again…”
Like tears in rain.
In 1964, Sam Harris, who had some firmly big function even if we didn’t know what, arrnged for Arecibo to be used a couple of times for amateur radio moon bounce. The dish so big that it meant many with small antennas could get through. This was early in Arecibo’s life, so either the time wasn’t all booked up, or it was allocated to “testing”.
So they heard a lot of signals, crowded like one of the shortwave ham bands. And they had one recorder which was part of the radio telescope, so after the fact they could tune up and down the band with a receiver to hear individual stations. But yes, the point where they were recording had to be “wide” or else they’d have only been recording one frequency.
Michael
Ok, I know of the first one in this list. Who are you talking about?
https://en.wikipedia.org/wiki/Sam_Harris_(disambiguation)
He’s not on that list.
As a ham, he was big, he did moonbounce early on and long wrote a column on VHF in QST. He also had the first article in a ham magazine about the parametric amplifier, about 1958, very important for low noise reception before transistors. But while he was alive, not much was said about what he did professionally.
In more recent years, it looks like he invented the parametric amplifier, certainly made it a practical thing, but he was working at Microwave Associates at the time, and they got the patent. I always had the impression he was a “good technician” at Arecibo, but in trying to track things down in recent years, he may have been “Chief Engineer” at the radio telescope.
Once he moved to Arecibo, it seemed like he was busy with that, so for the last ten or so years of his life, he wasn’t doing much in ham radio, at least not where someone would write about it. He worked at a major “toy” and getting those distant signals was probably at least a ten fold increase on doing amateur radio moonbounce.
He died in the seventies, somewhere round 1975.
Michael
In the late sixties I was a member of the National Radio Club, an AM broadcast DX’ers group. As I remember we published a story talking about exactly this ability. We did it as an April Fool’s joke!
“If it was important enough to be worth the expense and complexity of broadcasting it on the radio, it’s probably worth keeping for future reference.”
Or not. I was a tech at commercial AM stations in the 70s and 80s, and honestly, it might be fun to hear an hour or two of what Top 40 radio sounded like… but that’s enough thanks. I think most commercial radio these days is even less noteworthy, and with syndication you have many stations airing exactly the same program.
Commercial radio today is still mostly audio wallpaper, and I don’t think the world really needs every scrap of wallpaper to be faithfully archived. The truly great radio content (eg NPR and the better national broadcasters – BBC etc) are already properly archived… and then there’s podcasts.
The vast majority of pop music is archived, and probably always will be, on commercial sites that sell it. So for commercial radio you’d basically be recording the inter-song DJ blather, AKA the least informative thing it is possible to transmit across any medium.
But let’s not forget that FM radio had some nice nuggets of “Album Oriented Rock” & “Progressive Rock”, back in the early 70’s.
( A wiki link for the youngsters amongst us).
https://en.wikipedia.org/wiki/Album-oriented_rock
What I’d love to go back and harvest would be the rare performances and obscure mixes that some Disc Jockeys somehow managed to acquire or make.
There’s a wicked funky jazz version/mix of The Blackbyrds “Rock Creek Park” That I’ve (unsuccessfully) been trying to find for years.
Sadly, the DJ who possesed it, seems to have taken that reel of tape to the grave with himself.
Rock greek park? You mean this one https://www.whosampled.com/The-Blackbyrds/Rock-Creek-Park/
Right song but not the version I’m looking for.
Thanks for taking the time to look though!
I’ve crawled through a bunch of uploads on youtube and a few other websites , hoping to identify the version.
The one I’m after has no vocals in the second half of the song. Rather it’s a bit heavy on the electronics (synthesizers mostly). a little bit like some of the stuff that Jan Hammer & Jeff Beck played together back in the 70’s.
I’m not sure there’s much variety in radio newscasts. In high school, I dropped by to see a friend who worked at a local AM station. The station’s “news” was simply what came over the AP wires. The same is probably true today.
On the other hand, recordings of talk radio, including NPR and listener call-ins, might be interesting.
Funny enough I had considered a project like this a few years back in the wee early days of RTL-SDR’s becoming popular. Essentially a fully user friendly web based service where you could simply dial in virtually any AM/FM station and ‘turn back the clock’ and listen to said station at any period of time. But the moment I started brainstorming it, the immediate thought hit me of copyright and related legal issues which is WAY out of my league.
However if anyone starts working on such a project, I’d be more than willing to help supply ‘content’ as it were or provide any help I can personally. I think it’d even be nice and a fun side project to acquire old recordings that could be reasonably date/time stamped and merge them into such a timeline so the time machine aspect could be expanded backwards as far as possible.
At the end of the talk somebody did ask if there were any copyright concerns about offering this content up since obviously some of it is going to be commercial music stations.
The consensus seemed to be that since they weren’t making any money off of the project and that it was freely broadcast they don’t expect anyone to give them a hard time. Especially since all the commercials that originally paid for it are still there.
But definitely an untested gray area.
Awesome! Would be great to have a resource for live satellite A/V broadcast feeds too.
Hey there. On tom’s website, I only see 1986 and 1991. I’d love to see more broadcasts available. If someone could hook me up with some more band spectrum files I’d be amazed. I’ve made some interesting vhf band spectrum recordings, like recording the entire weather radio band as it sounded from a massive band opening earlier this year on July 26th, at 3 in the morning hearing Wichita Kansas, at 3 in the morning from Missouri Valley, Iowa where I’m located.
I’d like to get some good spectrum recordings of NWR from before 2017, maybe 2016 and older. But I’d really like to get some good AM Band spectrum recordings. Specificly during 199, may 3, during the famous Oklahoma Tornado OUtbreak. Would be very maazing to hear how it all sounded in various spots.
Forgive some of the misspelling the keyboard was acting up Lol.