Where the music matters

Posted on Sat 05 March 2022 in Technology

Invisible airwaves crackle with life
Bright antenna bristle with the energy
Emotional feedback on timeless wavelength
Bearing a gift beyond price, almost free

__

One likes to believe in the freedom of music
But glittering prizes and endless compromises
Shatter the illusion of integrity, yeah

I enjoy listening to music on the radio. You hear new things, and it can make for a shared experience you can discuss with friends. I like some DJs. Some DJs drive me nuts, especially the ones with "DJ voice" where there are odd sing-songy cadences and intonations.

I really don't like ads. Especially the ones that become earworms taking up uninvited space in my brain.

I value offline listening. I don't always have internet or FM reception. I value audio quality and fidelity and try to maximize it when possible. Life is too short to listen to crappy audio.

As personal technology project I made an automated scheduled digital recording of FM radio and audio file processing that does the following:

  • Identifies the start and end time of voice and music segments
  • Removes the voice segments, leaving only music
  • Encodes the music as MP3 files, including some arbitrary fixed duration track segmentation so I can skip songs when I like
  • Tags the MP3 files with arbitrary artist, date, and track numbers based on the time of day the recording was made
  • Retains a rolling library of the past week of recordings

The one thing I don't have is any real song metadata. No artist, album, genre, track names. With additional effort I think this would be possible to accomplish, but in the meantime Shazam exists when I need to know what I'm listening to.

This is all solely for personal use and I don't share or distribute the recordings. Fair use time-shifting, just like your old VCR or cassette deck did.

Methods of FM radio recording

I mentioned quality matters to me. Seattle has some challenges with FM radio reception, mostly multipath fragmentation/reflection due to the hills, or that's my layman's understanding. This manifests as hiss.

I have a rooftop antenna that has been up there for over 20 years. I use it with SageTV and the Comskip plugin to record local OTA HD television and remove the ads. We don't watch a lot of local TV these days, but it still comes in handy from time to time. Depending on your situation a rooftop antenna is probably overkill, but I already had it so I used it.

I tested three methods of digital FM recording:

  1. Recording the analog output of a respected vintage FM tuner, the Marantz ST6000. I record it using a Raspberry Pi with a HiFiBerry DAC+ ADC. fmtunerinfo.com is a great site to learn about different FM tuners.
  2. Recording the "regular" FM broadcast using a USB software defined radio device and a Raspberry Pi
  3. Recording the digital FM broadcast, misleadingly branded as HD Radio, using a USB software defined radio device and a Raspberry Pi

All three methods work great and produce very listenable recordings. Each has pros and cons.

For method #1 with the external FM tuner and ADC, the results are only as good as the tuner and antenna can make them. If there is multipath or hiss, there isn't much you can do. I didn't really explore processing the recordings to try to remove hiss, but that is possible. The HifiBerry ADC is excellent and is overkill for this source. It puts very minimal load on the Raspberry Pi, but this solution does require a Raspberry Pi while the others don't: for the other two methods you could use an old laptop or run it on an existing desktop computer you may already own. You also have to have the tuner on all the time, or put it on its own power timer, and if you want to record more than one station you need to manually switch stations on the tuner. But, this does have the advantage of being the simplest method from the perspective of setting up the Raspberry Pi. You don't need to clone any github repositories or compile or "make" the software yourself. These aren't difficult things and they are all pretty well documented, but that's a hurdle for some people. If you want to avoid all that this is technically the simplest to get running. Well, it is simplest from the recording perspective, but the subsequent processing for any of these recording methods requires a bit of heavy lifting to build and setup the software.

Methods 2 and 3 require a SDR or software defined radio device. Software defined radios are amazing. One of my personal goals with this project was just to put my toes in that water. You can listen to or record all kinds of stuff including pagers, police and fire radio, unencrypted baby monitors, weather faxes, marine AIS data, shortwave and ham radio...you name it, if it is whizzing through the air you can listen or record it. And the more amazing thing is you can do it all simultaneously: you can record all the FM radio broadcasts of the whole dial at one time. You would need a lot of storage to do that...but some people do it. I've read of some people who will take their SDR and a laptop to a location that has excellent reception, capture a whole range of frequencies to a hard disk, and take it home to their apartment where they get no reception and process and listen to it. And many SDRs are really quite affordable. The most popular SDR is the RTL-SDR.com model that started it all and can be had for about $30 on Amazon, or under $60 with a nice starter antenna kit. There are better models. FM recording is pretty much kindergarten for SDR enthusiasts and YAGNI applies. A step up would be the Noelec NESDR v4 which is about $34. A major step up, and required for my prefered recording method, is the Airspy HF+ Discovery (available from Airspy.us) which is $170. Top of the line is the HackRF from Great Scott Gadgets, who is a frequent guest on one of my favorite podcasts, The Amp Hour. But it is over $300 and super overkill for this project. I don't have one of these.

SDRs are amazing but some of the software is a little on the bleeding edge. I had a few re-dos on setting up the Raspberry Pi to make it all work in a headless method with no human intervention required for scheduled recordings. There is a little learning curve. Some of the support and community forums are populated by experts who live and breathe this stuff and are not always friendly to newbs. More than once I was scolded with comments like "it sounds like you are just blindly copying and pasting things you found on the internet and don't know what you are doing" (true!) and "if you want to do that, you are going to need to code some C++" (untrue!). It is all possible with existing open source software, and it does all work great once you get it running.

For method #2 I am using the Airspy SDR, mostly because it supports a really awesome piece of software that features some effective FM radio multipath handling to minimize the hiss. The library is here: https://github.com/jj1bdx/airspy-fmradion and is mostly maintained by some Japanese folks who seem super knowledgeable about all this stuff. Much respect to Kenji Rikitake and the other contributors. A wonderful example of the power of open source software. That library only works with Airspy SDRs, so you need to pony up for the $170 Airspy SDR. Maybe the $99 model will work too? This is the method I settled on for my personal use. The software has a long list of dependencies you need to compile and install. It was by far the most work to install, including installing VOLK, the "Vector-Optimized Library of Kernels" which was a bit of a bear. The installation instructions are sound and you will get there if you persist. And you still need a good antenna setup as you are starting with the regular old analog over the air FM broadcast.

Method #3 is probably the sweet spot for most people. With this method I recorded the digital "HD Radio" broadcast that is sent over the air. Since it is a digital source, it has the major advantage of being entirely hiss-free. But HD Radio is just a marketing label. There is absolutely nothing High Definition about HD Radio. It tops out at about 96kbps. For me, it had the major disadvantage of compression artifacts that I could not un-hear once I noticed them. Unacceptable! It also has a hard requirement that you can only record a station that broadcasts HD Radio (not all do), and you must have a good enough signal and antenna to lock onto the digital HD Radio signal. Not hard if you are in a city and put a minimal effort into setting up your antenna. You could also tap into the song metadata that is included in these digital broadcasts to tag your MP3 files with real artist and song information, but I have not gone to that effort. The critical piece of software required to do this is https://github.com/theori-io/nrsc5 which will work the the RTL-SDR SDR and any of its ilk, including the NOOELEC models.

I'm not going to get into the weeds of all the installation steps. The software installation instructions mostly work well as documented. Set aside a few hours for that VOLK tuning to run once it is installed. Feel free to get in touch with me if you get stuck. Overall I'd say get your SDR working at a basic level with its drivers installed and responding to simple diagnostic commands before trying to install the NRSC5 library of the Airspy-FMRadion library.

Scheduled recordings

My goal was scheduled recordings using cron that just run on a headless (no monitor or keyboard or mouse) Raspberry Pi with no daily intervention required. It is a success. Such a success that I struggled to even remember the hostname I setup on the Raspberry Pi to get back into it and recall exactly how I had it setup so I could write this section of this blog post. It just runs.

This is the script that I run as a daily cron job to record using the Airspy SDR with the airpspy-fmradion library:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
#!/bin/bash

for i in 1 2 3  
do  
 today=`date '+%m-%d'`;  
 filename="/home/pi/recording/showname-${today}-${i}.wav"  
 timeout 3600 /usr/local/bin/airspy-fmradion -m fm -t airspyhf -q -c freq=90300000,srate=768000,hf_att=0 -U -E36 -W $filename  
 cp $filename /mnt/files/radio/recorded  
 rm $filename  
done

This produces three one-hour long wav files. The files have a filename like showname-date-1.wav (-2 for the second hour, -3 for the third hour). The file naming convention drives subsequent processing and the MP3 tags.

That's it for the recording. Now the magic happens!

Processing recordings

When I initially completed this project and published this article, I was using the Ina Foss Speech Segmenter library to detect voice versus music. This software was overkill for my needs as it also classifies speech by gender, and is quite resource-intensive. Since then I have switched to a Voice Activity Detector (VAD) library that, as you would expect, just classified voice vs. "not voice." Hot dog or not hot dog. DJ/Ad or music. The library I am using is https://github.com/snakers4/silero-vad which has 8.2k stars and runs much faster and leaner than the Ina Foss library. It might even run OK on the Raspberry Pi, which the Ina Foss library never would.

A cron job on this VM kicks off 6 minutes after each hour, to allow the recordings to finish before it starts. It just picks up whatever files it finds, processes them, and then cleans up.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
#!/bin/bash

find /tank/music/radio* -atime +7 -exec rm {} \;

/tank/pat/projects/valero/bin/python /tank/pat/projects/valero/vad.py

for f in /tank/files/kexp/recorded/*.edit; do
 file=$(basename $f)
 wav="${file%.*}"
 base="${wav%.*}"
 echo "wav ${wav} base ${base}"
 i=1
 while read edit; do
  tracknum=$(printf "%02d" $i)
  /usr/bin/sox ${f%.*} /tank/files/kexp/split/${base}-${tracknum}.wav trim ${edit}
  i=$((i+1))
 done <$f
 mv $f /tank/files/kexp/archive
 mv /tank/files/kexp/recorded/${base}.wav /tank/files/kexp/archive
done

for f in /tank/files/radio/split/*.wav; do  
 file=$(basename $f)  
 wav="${file%.*}"  
 base="${wav%.*}"  
 echo "wav ${wav} base ${base}"  
 /usr/bin/ffmpeg -i "$f" -vn -ar 44100 -ac 2 -channel_layout stereo -b:a 192k /tank/music/radio/"${base}.mp3";  
done  
mv /tank/files/radio/split/* /tank/files/radio/archive

/tank/pat/projects/valero/bin/python /tank/pat/projects/valero/tagger.py

find /tank/files/tag/archive* -mtime +10 -exec rm {} \;

There is a lot going on in the file /tank/pat/projects/valero/vad.py and I will be honest in that I leaned quite a bit on LLMs to create it. I think I am going to put all of the code for this project into a Github repository rather than embed it here.

I break it down as:

  1. Delete all of the final processed mp3 files older than a week
  2. Run the recorded wav files through Silero VAD python library. It produces a series of time-coded segments identifying speech vs. music.
  3. The first for loop does some munging of the VAD output to create commands for sox, a general purpose audio utility that can cut wave files up based on start time and duration.
  4. The second for loop takes those wav files that now contain only music and encodes them to mp3 files using ffmpeg
  5. The tagger.py script tags the mp3 files with the show name, radio station, date, and the tracks are just named track 1, track 2, and so on

For the MP3 ID3 tags I use a bogus genre of Radio as this makes it easier to pick these out from my media player client software.

Results

Did I get what I expected? That's an interesting question. I achieved the technical goals, but I honestly have some mixed feelings about the results.

Radio is kind of special, and I've effectively neutered a component of it. I have zero remorse about removing the ads and fundraiser promos and endless talk about the healing power of music. And there are a few DJs whose voices I am glad to be rid of.

It is very nice to be able to skip tracks, and whole shows that just don't fit my vibe at a given point in time. I'm not always in the mood to be taken on a musical journey or history lesson of the 1920's jazz roots of modern hiphop. Skip!

But I don't miss all the DJs. I do have some favorite DJs and I do miss them. I miss knowing what it is I'm listening to, and some of the context for how it was chosen. There is also an immediacy of listening to radio live and knowing that you are listening to the same thing others are hearing at that moment. My book club read The Beastie Boys book and there is a whole piece at the start where they talk about the power of FM radio in NYC where you could travel around the city and have the sounds of specific radio stations follow you around, coming out of open apartment and car windows and boomboxes. Sort of like how some people cannot watch a recorded sporting event, the loss of this shared experience is...something.

But after living with this setup for a couple of years now, overall I love it!

Technology