#193 SR_ERROR_INVALID_METADATA error is back in version 1.64
Streamripper 1.64.6 refuses to rip many radio stations with the error SR_ERROR_INVALID_METADATA.
Here is an example:
$ streamripper http://dradio-mp3.t-bn.de/dlf128k_live
Connecting...
stream: Deutschlandfunk - MP3
server name: Icecast 2.3.2
declared bitrate: 128
meta interval: 16000
[skipping... ] - [ 1kb]
error -28 [SR_ERROR_INVALID_METADATA]
bye..
shutting down
The example radio station does not send streamtags. Anyway, Streamripper should be capable to rip such stations.
Streamripper 1.63.5 (as in Ubuntu 9.04) worked with such radio stations:
$ streamripper-1.63.5 http://dradio-mp3.t-bn.de/dlf128k_live
Connecting...
stream: Deutschlandfunk - MP3
server name: Icecast 2.3.2
bitrate: -1
meta interval: 16000
skipping... ] - [ 157kb]
Probably Streamripper 1.64 introduced the old SR_ERROR_INVALID_METADATA problems again. I compiled and tested both versions without additional patches and with the options "--with-included-libmad=no" and "--with-included-tre=no" on openSUSE 11.1.
Yeah, I don't know. It works fine for me...
gsharp@stonesmith:~/tmp$ streamripper --version
Streamripper 1.64.6
gsharp@stonesmith:~/tmp$ streamripper http://dradio-mp3.t-bn.de/dlf128k_live
Connecting...
stream: Deutschlandfunk - MP3
server name: Icecast 2.3.2
declared bitrate: 128
meta interval: 16000
[skipping... ] - [ 1.27M]^C
shutting down
bye..
Are you using a proxy, or any special network connection? Are you on 64-bit machine? What do you see in debug log?
No, I use a 32Bit system.
But yes, I use a proxy server. It is Apache 2.2.10 with mod_proxy mod_proxy_connect mod_proxy_ftp and mod_proxy_http.
The radio stream http://dradio-mp3.t-bn.de/dlf128k_live works fine with media players like Kaffeine. I attached the debug log for these commands:
$ export http_proxy=http://localhost:80/
$ streamripper http://dradio-mp3.t-bn.de/dlf128k_live --debug
Connecting...
stream: Deutschlandfunk - MP3
server name: Icecast 2.3.2
declared bitrate: 128
meta interval: 16000
[skipping... ] - [ 1kb]
error -28 [SR_ERROR_INVALID_METADATA]
bye..
shutting down
If the proxy server is disabled (unset http_proxy) Streamripper works fine.
So probably Apache corrupts the MP3 stream. How can we further debug this?
Debug log for Streamripper with Apache proxy
The metadata is coming 83 bytes too late. You can see it right after "METADATA TITLE".
I wonder if the proxy is adding an http header or something to the icy connection. Do you get any output file? If not, try using the -a option to get a showfile. Then, run "od" to look at the first few lines. What do you see?
Example of "od" command:
od -t x1 -c sr_program_2009_04_27_13_07_50.mp3 | head -n 10
My Streamripper does not produce a file, if it is used together with the
Apache proxy.
But Wireshark (network protocol analyzer) helped me. This are the HTTP
headers of the connection between Streamripper and the Apache proxy:
GET http://dradio-mp3.t-bn.de:80/dlf128k_live HTTP/1.1
Accept: */*
Cache-Control: no-cache
User-Agent: Streamripper/1.x
Icy-Metadata: 1
Connection: close
Host: dradio-mp3.t-bn.de:80
HTTP/1.1 200 OK
Date: Mon, 27 Apr 2009 17:25:30 GMT
Server: Icecast 2.3.2
Content-Type: audio/mpeg
icy-br: 128
icy-description: Deutschlandfunk - MP3
icy-genre: www.dradio.de
icy-name: Deutschlandfunk - MP3
icy-pub: 1
icy-url: www.dradio.de
Cache-Control: no-cache
icy-metaint: 16000
Via: 1.1 mybox
Connection: close
Transfer-Encoding: chunked
578
[...]MP3data[...]
And this are the HTTP headers of the connection between the proxy
and the Icecast radio server:
GET /dlf128k_live HTTP/1.1
Host: dradio-mp3.t-bn.de
Accept: */*
Cache-Control: no-cache
User-Agent: Streamripper/1.x
Icy-Metadata: 1
Via: 1.1 mybox
Connection: Keep-Alive
HTTP/1.0 200 OK
Content-Type: audio/mpeg
icy-br:128
icy-description:Deutschlandfunk - MP3
icy-genre:www.dradio.de
icy-name:Deutschlandfunk - MP3
icy-pub:1
icy-url:www.dradio.de
Server: Icecast 2.3.2
Cache-Control: no-cache
icy-metaint:16000
[...]MP3data[...]
The proxy server responds with a chunked encoded stream. This is
allowed, because Streamripper requested a HTTP/1.1 session.
Unfortunately Streamripper currently does not seem to understand the
chunked encoding. Therefore it misinterprets the MP3 streams and does
not find any correct metadata. Freshclam (the updater of Clamav) had a
similar problem: https://wwws.clamav.net/bugzilla/show_bug.cgi?id=431
I see two solutions:
1) Streamripper should request with the "HTTP/1.0" keyword, if it only
understands the normal HTTP/1.0 encodings.
2) Streamripper should implement the chunked encoding of HTTP/1.1.
Solution 1) is trivial. A patch is attached. With this patch I could
connect to my radio station.
Solution 2) causes some more changes, but is probably more forward-looking.
Patch for solution 1 - HTTP/1.0
Nice detective work. I agree with your conclusions.
The fix should be option (2), because some servers do pattern matching of http headers as an anti-ripping measure. If you compile from source, you may like to modify line 231 of http.c.
Do you know, if new versions of streamripper are planned? It is probably not so hard to write a HTTP/1.1 chunked parser, but it will be useless if nobody includes the patch into a new streamripper version.
Joint Radio Beat cannot be used with streamripper:
$ streamripper -v
Streamripper 1.64.6
$ streamripper http://star.jointil.net/proxy/jrn_beat?mp=/stream -u "Mozilla/5.0+(Windows+NT+5.1;+WOW64;+x64;+rv:52.0)+Gecko/20100101+Firefox/52.0"
Connecting...
stream: Joint Radio Beat
server name: SHOUTcast/
declared bitrate: 192
meta interval: 8192
[skipping... ] - [ 1kb]
error -28 [SR_ERROR_INVALID_METADATA]
bye..
shutting down
Adding --debug didn't output anything extra.
Last edit: David Hedlund 2017-09-21
It is nearly the same problem which I discovered. Your radio station sends HTTP/1.1 content with chunked encoding and streamripper does not understand the chunked encoding:
But with my patch you can downgrade streamripper to HTTP/1.0 and streamripper will work with your radio station.
Thanks.
Can you please have a look at SR_ERROR_CANT_RESOLVE_HOSTNAME - https://sourceforge.net/p/streamripper/bugs/223/ and report the cause of it if you find it?
Last edit: David Hedlund 2017-09-22
See also: HTTPS error - https://sourceforge.net/p/streamripper/bugs/224/
Last edit: David Hedlund 2017-09-22
OK, I commented both bugs.
Can you merge in the fix to master?
Anyone on Windows facing this can modify streamripper 1.64.6's
streamripper.dlland patchHTTP/1.1toHTTP/1.0. Good luck!any way to patch streamripper on OSX (the version installed via homebrew in terminal)?
A radio program I've been time-delaying for convenient listening for years, just recently broke with this error. I have streamripper recording simultaneously on geographically independent machines, and they all broke at once, so I knew it was a source change. (I tried installing a Windows version on my desktop, it's broken as well.) I'm running v1.64.6 on FreeBSD.
The good news, is that I applied the fix above and it worked. Instructions:
su (password)
cd /usr/ports/audio/streamripper
./configure
cd work/streamripper-1.64.6
edit lib/http.c, search for "1.1". Find two blocks, one with "HTTP/1.0", followed by almost identical block with "HTTP/1.1". It's an 'sprintf' in the function http_construct_sc_request(). Move the line "#if defined (commentout)" from the former block to immediately before the latter block. Move the corresponding "#endif" to immediately after the latter block.
make
make install
FYI: I tried simply adding -Dcommentout in config, but this broke other stuff at compile time.
How do I apply this patch file? I am running this on Debian and still feel pretty noobish. Basically i have not clue what to do with it...
So crap that this is not maintained. I hacked the binary, too lazy to compile everything from source. It works for me on Ubuntu. Be sure to give the file eXecution rights. I hacked the original /bin/streamripper (Ubuntu 22.04)
Last edit: G.G 2023-01-24
thanks a lot ! A REALLY BIG THANK YOU
Even with this HTTP 1.0 patch still some streams do not work. Not sure what it causes exactly, but it seems still related with dash (chuncked) streams, creating loads of files in the output directory. My guess: One file for each chunk.
On Windows machines you can best use Streamwriter. It is a GUI with schedule options. Used it for many years while my machine was on for 24/7. But right now I rather use a less powerconsuming option, like a remote VPS or (local) Rasp.Pi.
For commandline Linux I would sugest using yt-dlp. I am experimenting with it, because it is very capable of downloading chuncked and non-chuncked streams. For scheduling you need to write a script and use CRON, but using Streamripper you might already did that.
En example of a 2 hours and 6 minutes recording. You can CRON this daily, weekly, whatever you like. Not thourougly tested, but seems to work fine.
yourscriptname.sh
run/schedule with
In the end the file might be something like 30 seconds longer, probably because of the buffering.
Of course you need to create the output directory and be sure it is writable.
Last edit: G.G 2023-02-01
Instead of downloading the patch in the blog, compiling, etc; an alternative is directly editing a copy of the binary file using a tool such as ghex!
Although, I do dabble in compiling from source code, upon seeing what the patch was, I decided to rely on a hacking trick I employed some decades ago; for this situation ..
Change [user]:[user] to match your user id and group id.
Find 'GET %s HTTP/1.1' and change the last '1' to '0'
Save the file
Change the permission back to root:root
Use the /usr/bin/streamripper-HTTP1.0 instead for the sites that fail with an error -28 "SR_ERROR_INVALID_METADATA"
This worked in Ubuntu 20.04 with Streamripper 1.64.6. This worked for one website that was giving me the above error. It may or may not work for your situation ...
Enjoy!
I confirm this workaround works. I used Okteta and used Find > CHAR to search for HTTP/1 to search for the HTTP/1.1 inside /usr/bin/streamripper which I changed into HTTP/1.0. This workaround is safer than downloading a patch from unvalidated sources.
Hello, can someone explain to me how to use the patch-file? That would be very kind of you.