Invisible airwaves crackle with life
Bright antenna bristle with the energy
Emotional feedback on timeless wavelength
Bearing a gift beyond price, almost freeOne likes to believe in the freedom of music
But glittering prizes and endless compromises
Shatter the illusion of integrity, yeah
I enjoy listening to music on the radio. You hear new things, and it can make for a shared experience you can discuss with friends. I like some DJs. Some DJs drive me nuts, especially the ones with "DJ voice" where there are odd sing-songy cadences and intonations.
I really don't like ads. Especially the ones that become earworms taking up uninvited space in my brain.
I value offline listening. I don't always have internet or FM reception. And while I am a middle aged gen X dude who suffered permanent hearing loss during Seattle's Grunge era (too many nights at Squid Row, the Offramp, the Vogue, and then there was that one Big Black show at the Georgetown Steamplant...), I value audio quality and fidelity and try to maximize it when possible. Life is too short to listen to crappy audio.
As an experimental personal technology project I am making automated scheduled digital recordings of FM radio and putting the recordings through a software pipeline that does the following:
The one thing I don't have is any real song metadata. No artist, album, genre, track names. With additional effort I think this would be possible to accomplish, but in the meantime Shazam exists when I need to know what I'm listening to.
I'm an Eagle Scout. I try to live my life by the Scout Oath and Law. Trustworthy, loyal, helpful, etc. Do a good turn daily. This is important to me. I think it is important when you are doing something different to look in the mirror and ask "am I the asshole?", especially when you are applying your privilege as a technologist to do something with someone else's intellectual property.
I've built this in a way where I am not burdening the FM radio stations with anything I am doing. Streamripper exists and I could rip digital streams from the internet. I'm not doing that. I am only recording from the free over-the-air FM broadcast.
I also decided that since I am denying the stations of one member of their advertising audience, it is important from an ethical perspective to do something for them in return. So I became a monthly donor to the station I record.
I am not a lawyer, but I am pretty confident this is legal in the United States. See the Audio Home Recording Act.
I also do this solely for my personal use. I do not distribute or share my recordings in any way. If you find this project interesting, you can probably replicate it for under $50 in parts.
I mentioned quality matters to me. Seattle has some challenges with FM radio reception, mostly multipath fragmentation/reflection due to the hills, or that's my layman's understanding. This manifests as hiss.
I have a rooftop antenna that has been up there for over 20 years. I use it with SageTV and the Comskip plugin to record local OTA HD television and remove the ads. We don't watch a lot of local TV these days, but it is still nice for Saturday Night Live. Depending on your situation a rooftop antenna is probably overkill, but I already had it so I used it.
I tested three methods of digital FM recording:
All three methods work great and produce very listenable recordings. Each has pros and cons.
For method #1 with the external FM tuner and ADC, the results are only as good as the tuner and antenna can make them. If there is multipath or hiss, there isn't much you can do. I didn't really explore processing the recordings to try to remove hiss, but that is possible. The HifiBerry ADC is excellent and is overkill for this source. It puts very minimal load on the Raspberry Pi, but this solution does require a Raspberry Pi while the others don't: for the other two methods you could use an old laptop or run it on an existing desktop computer you may already own. You also have to have the tuner on all the time, or put it on its own power timer, and if you want to record more than one station you need to manually switch stations on the tuner. But, this does have the advantage of being the simplest method from the perspective of setting up the Raspberry Pi. You don't need to clone any github repositories or compile or "make" the software yourself. These aren't difficult things and they are all pretty well documented, but that's a hurdle for some people. If you want to avoid all that this is technically the simplest to get running. Well, it is simplest from the recording perspective, but the subsequent processing for any of these recording methods requires a bit of heavy lifting to build and setup the software.
Methods 2 and 3 require a SDR or software defined radio device. Software defined radios are amazing. One of my personal goals with this project was just to put my toes in that water. You can listen to or record all kinds of stuff including pagers, police and fire radio, unencrypted baby monitors, weather faxes, marine AIS data, shortwave and ham radio...you name it, if it is whizzing through the air you can listen or record it. And the more amazing thing is you can do it all simultaneously: you can record *all* the FM radio broadcasts of the whole dial at one time. You would need a lot of storage to do that...but some people do it. I've read of some people who will take their SDR and a laptop to a location that has excellent reception, capture a whole range of frequencies to a hard disk, and take it home to their apartment where they get no reception and process and listen to it. And many SDRs are really quite affordable. The most popular SDR is the RTL-SDR.com model that started it all and can be had for about $30 on Amazon, or under $60 with a nice starter antenna kit. There are better models. FM recording is pretty much kindergarten for SDR enthusiasts and YAGNI applies. A step up would be the Noelec NESDR v4 which is about $34. A major step up, and required for my prefered recording method, is the Airspy HF+ Discovery (available from Airspy.us) which is $170. Top of the line is the HackRF from Great Scott Gadgets, who is a frequent guest on one of my favorite podcasts, The Amp Hour. But it is over $300 and super overkill for this project. I don't have one of these.
SDRs are amazing but some of the software is a little on the bleeding edge. I had a few re-dos on setting up the Raspberry Pi to make it all work in a headless method with no human intervention required for scheduled recordings. There is a little learning curve. Some of the support and community forums are populated by experts who live and breathe this stuff and are not always friendly to newbs. More than once I was scolded with comments like "it sounds like you are just blindly copying and pasting things you found on the internet and don't know what you are doing" (true!) and "if you want to do that, you are going to need to code some C++" (untrue!). It is all possible with existing open source software, and it does all work great once you get it running.
For method #2 I am using the Airspy SDR, mostly because it supports a really awesome piece of software that features some effective FM radio multipath handling to minimize the hiss. The library is here: https://github.com/jj1bdx/airspy-fmradion and is mostly maintained by some Japanese folks who seem super knowledgeable about all this stuff. Much respect to Kenji Rikitake and the other contributors. A wonderful example of the power of open source software. That library only works with Airspy SDRs, so you need to pony up for the $170 Airspy SDR. Maybe the $99 model will work too? This is the method I settled on for my personal use. The software has a long list of dependencies you need to compile and install. It was by far the most work to install, including installing VOLK, the "Vector-Optimized Library of Kernels" which was a bit of a bear. The installation instructions are sound and you will get there if you persist. And you still need a good antenna setup as you are starting with the regular old analog over the air FM broadcast.
Method #3 is probably the sweet spot for most people. With this method I recorded the digital "HD Radio" broadcast that is sent over the air. Since it is a digital source, it has the major advantage of being entirely hiss-free. But HD Radio is just a marketing label. There is absolutely nothing High Definition about HD Radio. It tops out at about 96kbps. For me, it had the major disadvantage of compression artifacts that I could not un-hear once I noticed them. Unacceptable! It also has a hard requirement that you can only record a station that broadcasts HD Radio (not all do), and you must have a good enough signal and antenna to lock onto the digital HD Radio signal. Not hard if you are in a city and put a minimal effort into setting up your antenna. You could also tap into the song metadata that is included in these digital broadcasts to tag your MP3 files with real artist and song information, but I have not gone to that effort. The critical piece of software required to do this is https://github.com/theori-io/nrsc5 which will work the the RTL-SDR SDR and any of its ilk, including the NOOELEC models.
I'm not going to get into the weeds of all the installation steps. The software installation instructions mostly work well as documented. Set aside a few hours for that VOLK tuning to run once it is installed. Feel free to get in touch with me if you get stuck. Overall I'd say get your SDR working at a basic level with its drivers installed and responding to simple diagnostic commands before trying to install the NRSC5 library of the Airspy-FMRadion library.
My goal was scheduled recordings using cron that just run on a headless (no monitor or keyboard or mouse) Raspberry Pi with no daily intervention required. It is a success. Such a success that I struggled to even remember the hostname I setup on the Raspberry Pi to get back into it and recall exactly how I had it setup so I could write this section of this blog post. It just runs.
This is the script that I run as a daily cron job to record using the Airspy SDR with the airpspy-fmradion library:
#!/bin/bash
for i in 1 2 3
do
today=`date '+%m-%d'`;
filename="/home/pi/recording/showname-${today}-${i}.wav"
timeout 3600 /usr/local/bin/airspy-fmradion -m fm -t airspyhf -q -c freq=90300000,srate=768000,hf_att=0 -U -E36 -W $filename
cp $filename /mnt/files/radio/recorded
rm $filename
done
This produces three one-hour long wav files. The files have a filename like showname-date-1.wav (-2 for the second hour, -3 for the third hour). The file naming convention drives subsequent processing and the MP3 tags.
That's it for the recording. Now the magic happens!
The processing pipeline can't run on the Raspberry Pi. I tried. The key piece of the puzzle is a nifty python library I found that uses TensorFlow machine learning with a pre-trained corpus to do the segmentation of voice vs. music. And boy that thing is resource-intensive. Maybe you can get it running on the Pi if you chop the recordings up into smaller lengths than one hour. I ran into the problem of needing it to be 64 bit in order to allocate enough memory, which is a non-starter on the Pi.
I happen to have a home server that I use for a handful of VMs fulfilling different utility purposes including our home SageTV DVR software, a Squeezebox music server, and a home security camera Windows VM running Blue Iris. But the biggest VM is a Ubuntu server with a RAID-Z2 ZFS file system for backup storage, photos, media, and general file storage. ZFS is a bit of a memory hog, so this VM has 32GB of RAM and 4 virtual processors allocated. The server has a Xeon Silver 4214 12 core processor with 64GB ECC RAM. Not exactly your every day piece of kit laying around waiting to be repurposed. I used what I had, and it can handle this load. You can probably get this working on a lesser rig like a spare laptop, but you will want 64 bit Ubuntu and a goodly amount of RAM.
There is a python library calleg Speechsegmenter that does this heavy lifting of identifying speech vs. music. Find it here: https://github.com/ina-foss/inaSpeechSegmenter
The cron job on this VM kicks off 6 minutes after each hour, to allow the recordings to finish before it starts. It just picks up whatever files it finds, processes them, and then cleans up.
#!/bin/bash
find /tank/music/radio* -atime +7 -exec rm {} \;
source "/home/pat/speechsegmenter/env/bin/activate"
nice -n 10 /home/pat/speechsegmenter/env/bin/python /home/pat/speechsegmenter/radio.py
for f in /tank/files/radio/recorded/*.edit; do
file=$(basename $f)
wav="${file%.*}"
base="${wav%.*}"
echo "wav ${wav} base ${base}"
i=1
while read edit; do
tracknum=$(printf "%02d" $i)
/usr/bin/sox ${f%.*} /tank/files/radio/split/${base}-${tracknum}.wav trim ${edit}
i=$((i+1))
done <$f
mv $f /tank/files/radio/archive
mv /tank/files/radio/recorded/${base}.wav /tank/files/radio/archive
done
for f in /tank/files/radio/split/*.wav; do
file=$(basename $f)
wav="${file%.*}"
base="${wav%.*}"
echo "wav ${wav} base ${base}"
/usr/bin/ffmpeg -i "$f" -vn -ar 44100 -ac 2 -channel_layout stereo -b:a 192k /tank/music/radio/"${base}.mp3";
done
mv /tank/files/radio/split/* /tank/files/radio/archive
/home/pat/speechsegmenter/env/bin/python /home/pat/speechsegmenter/tagger.py
find /tank/files/tag/archive* -mtime +10 -exec rm {} \;
I break all that down as:
Here is the radio.py Python script that does the segmentation:
from inaSpeechSegmenter import Segmenter
import os
seg = Segmenter()
directory = r'/tank/files/radio/recorded/'
for filename in os.listdir(directory):
if filename.endswith(".wav"):
segmentation = seg(directory + filename)
musicStart = 0
music = False
with open(directory + filename + '.edit', 'w') as editFile:
for s in segmentation:
label = s[0]
start = s[1]
end = s[2]
print(f"{label} from {start} to {end}")
if label != "male" and label != "female":
if music == False:
musicStart = start
music = True
if (end - musicStart > 120):
editFile.write(f"{musicStart} ={end}")
editFile.write("\n")
musicStart = end;
else:
if music == True and start - musicStart > 10:
editFile.write(f"{musicStart} ={start}")
editFile.write("\n")
musicStart = 0
music = False
and the tagger.py script, which uses the eyed3 python library to do mp3 tagging is:
import eyed3
import os
import urllib.request
from pathlib import Path
directory = r'/tank/music/radio/'
files = os.listdir(directory)
sorted_files = sorted(files)
lastshow="nothing"
lastday=0
track=1
for filename in sorted_files:
if filename.endswith(".mp3"):
basename = Path(filename).stem
print(filename)
audiofile = eyed3.load(directory + filename)
show=basename.split('-')[0]
day=basename.split('-')[2]
if show != lastshow:
lastshow = show
track=1
if day != lastday:
lastday = day
track=1
if show == "morning":
audiofile.tag.artist = "The Morning Show"
audiofile.tag.album = "MS " + basename.split('-')[1] + "-" + basename.split('-')[2]
if show == "afternoon":
audiofile.tag.artist = "The Afternoon Show"
audiofile.tag.album = "PM " + basename.split('-')[1] + "-" + basename.split('-')[2]
if show == "midday":
audiofile.tag.artist = "The Midday Show"
audiofile.tag.album = "Mid " + basename.split('-')[1] + "-" + basename.split('-')[2]
if show == "drivetime":
audiofile.tag.artist = "Drive Time"
audiofile.tag.album = "DT " + basename.split('-')[1] + "-" + basename.split('-')[2]
audiofile.tag.genre = "Radio"
audiofile.tag.album_artist = "Station"
audiofile.tag.title = "Track " + str(track)
audiofile.tag.track_num = str(track)
track=track+1
audiofile.tag.save()
contents = urllib.request.urlopen("http://192.168.0.196:9000/settings/index.html?p0=rescan").read()
Folks in Seattle have probably figured out what station I am recording by now.
I use a bogus genre of Radio as this makes it easier to pick these out from my media player client software.
The final line just hits a URL on my squeezebox server to rescan the media library. This way, the four squeezebox players scattered around my home all are up to date within about 20 minutes after an hourly recording file is produced.
This was a nontrivial project. I hope someone else gets some utility from it.
Did I get what I expected? That's an interesting question. I achieved the technical goals, but I honestly have some mixed feelings about the results.
Radio is kind of special, and I've effectively neutered a component of it. I have zero remorse about removing the ads and fundraiser promos and endless talk about the healing power of music. And there are a few DJs whose voices I am glad to be rid of.
It is very nice to be able to skip tracks, and whole shows that just don't fit my vibe at a given point in time. I'm not always in the mood to be taken on a musical journey or history lesson of the 1920's jazz roots of modern hiphop. Skip!
But I don't miss all the DJs. I do have some favorite DJs and I do miss them. I miss knowing what it is I'm listening to, and some of the context for how it was chosen. There is also an immediacy of listening to radio live and knowing that you are listening to the same thing others are hearing at that moment. My book club read The Beastie Boys book and there is a whole piece at the start where they talk about the power of FM radio in NYC where you could travel around the city and have the sounds of specific radio stations follow you around, coming out of open apartment and car windows and boomboxes. Sort of like how some people cannot watch a recorded sporting event, the loss of this shared experience is...something.
Current rating: 5 Share on Twitter Share on Facebook
Comments