mpg123

Hi folks,

there is a feature I worked a lot on in mpg123: The ability to produce gapless output from mp3 files, cutting away padding added by encoder and decoder machinery. That way, you can nicely play your ripped CD with seamless transitions (there are quite some CDs that are mixed continuously).

This feature relies on information that the encoder, well, LAME, places into a special frame at the beginning of the tracks. In essence, there is a header that tells mpg123 how much audio data it shall use out of the following MPEG stream.

Now, this leads to a problem when you happen to take such enhanced MP3 files and concatenate them together, like this:

	cat file1.mp3 file2.mp3 | mpg123 -

If file1.mp3 features such a special frame, mpg123 will stop producing output after hitting the end of the first file. While

	cat file1.mp3 file2.mp3 | mpg123 --no-gapless -

will give you the output of both files, as one might expect from MPEG audio. Incidentally, you got matching behaviour from playing simple WAV files with aplay on Linux:

	cat file1.wav file2.wav | aplay -

gives you only the sound of file1.wav.

Well, actually one does not have to see a problem here. WAV files have a header that tells the program how many bytes of audio data to use and that special frame in modern MP3 files does the same; both mpg123 and aplay just honor the information given and act accordingly. The problem is that for MPEG audio streams, traditionally, there is no such header -- even the concept of a "MP3 file" is not the initial idea. You have streams. Streams are concatenations of MPEG frames ... a stream produces as much decoded output as there are frames providing such.

Now this LAME header with the info for gapless decoding changes the meaning towards an MP3 file being more like WAV in that respect: One file is an entity and you cannot just throw two files together and expect it to work. If you do, one might call you naive. Well, people are naive. Or rather: ignorant.

Getting to the point now ... HTTP streams! Web radio producers are sometimes (?) not aware of this problem or don't expect clients to use the information in the way mpg123 does. Not for the first time, I found the situation that the radio stream starts with a jingle that contains such a header and then just throws in the actual radio stream. In such cases, mpg123 just plays the jingle and then thinks that the stream is over because the header told it so. People think mpg123 is broken because it doesn't play the actual music (or talk show, if listeners are so inclined;-).

You can fix that by disabling the gapless code:

	mpg123 --no-gapless http://my-funky-stream.tld

but the default for now would break on such streams.

The question: What to do about this?

I am asking both mpg123 users and developers. What would the senisble thing to do?

1. Should we just call the web radios broken?
2. Should the mpg123 app disable gapless decoding automatically for http streams?
	Note: That can trigger false positives in case you play individual files from a resource via HTTP.
3. Should the library try to detect the situation and resume decoding?
	May be possible ... when finding more MPEG frames than announced. But that may also trigger false positives.
4. Should mpg123 be more verbose about this and inform the user?
	"There is still data that may be MPEG audio but I stopped decoding because of gapless header info."

Thoughts on that? Further suggestions?


Alrighty then,

Thomas.

Re: [mpg123-users] Questions about gapless cutting on http streams or otherwise contatenated files. From: Tom Swirly <tom@sw...> - 2011-01-08 18:14 Attachments: Message as HTML      I've been idly wondering about similar issues, not in the same depth as you! > I am asking both mpg123 users and developers. What would the senisble thing > to do? > > 1. Should we just call the web radios broken? > They are in fact broken BUT my experience has been that if people are doing the wrong thing, telling them that they're broken doesn't ever get them to fix it up. I think calling them broken is fine - and might encourage people writing new radio software to do the right job - but if possible, mpg123 should try to do better. One interesting question might be - who's *writing* that broken software? It isn't the radio station providers - there must be some software developers doing this! If we could get to them and explain the issue, we might get better standards compliance in their next revisions. > 2. Should the mpg123 app disable gapless decoding automatically for http > streams? > Note: That can trigger false positives in case you play individual > files from a resource via HTTP. > 3. Should the library try to detect the situation and resume decoding? > May be possible ... when finding more MPEG frames than announced. > But that may also trigger false positives. > See below on these two. > 4. Should mpg123 be more verbose about this and inform the user? > "There is still data that may be MPEG audio but I stopped decoding > because of gapless header info." > Printing a message on the command line isn't even going to be seen by most users - but certainly better than nothing if it's easy! There might be an API possibility where you return a special code to the calling program and allow them to either continue reading or not... I don't personally understand the difference between cases 2 and 3... or is 3 simply an extension of 2? Either way, attempting to recover seems like a good idea. I've noted quite a few error messages from mpg123 and recoveries in my logs - I've been testing with all sorts of crappy files to see if my program works - and I've been very impressed with the fact that the library has never abnormally terminated, and in most cases I can't even hear an issue. Since good error recovery is apparently a feature of yours, recovering from this might be good. But, what are these false positives? What are the consequences? The fact that far more people listen to net.radio than play multiple files from http on a command line! So IMHO mpg123 should be concentrating on the net.radio listeners more. If a command line user occasionally gets an error message when playing multiple files from the command line, well, that's really not so bad. Thanks for all your hard work! > > Thoughts on that? Further suggestions? > > > Alrighty then, > > Thomas. > > > ------------------------------------------------------------------------------ > Gaining the trust of online customers is vital for the success of any > company > that requires sensitive data to be transmitted over the Web. Learn how to > best implement a security strategy that keeps consumers' information secure > and instills the confidence they need to proceed with transactions. > http://p.sf.net/sfu/oracle-sfdevnl > _______________________________________________ > mpg123-users mailing list > mpg123-users@... > https://lists.sourceforge.net/lists/listinfo/mpg123-users > > -- /t http://radio.swirly.com - art music radio 24/7 366/1000

Re: [mpg123-users] Questions about gapless cutting on http streams or otherwise contatenated files. From: Thomas Orgis <thomas-forum@or...> - 2011-01-11 08:17 Attachments: signature.asc      Am Sat, 8 Jan 2011 13:14:12 -0500 schrieb Tom Swirly <tom@...>: > > 1. Should we just call the web radios broken? > > > One interesting question might be - who's *writing* that broken software? > > It isn't the radio station providers - there must be some software > developers doing this! If we could get to them and explain the issue, we > might get better standards compliance in their next revisions. I tried to contact one of the affected radio stations (http://weta.org). Let's see if they can enlighten us. Also pointed to this discussion thread. > > 4. Should mpg123 be more verbose about this and inform the user? > > "There is still data that may be MPEG audio but I stopped decoding > > because of gapless header info." > There might be an API possibility where you return a special code to the > calling program and allow them to either continue reading or not... Ah, well, it is a question how much the libmpg123 library can do. Ususally, it's the program using the library that knows if there is web radio going on or not. Then it can just disable gapless code. Perhaps the library could try to mangle the default to off when it gets an ICY interval... but I'm not sure if this logic should go there. > I don't personally understand the difference between cases 2 and 3... or is > 3 simply an extension of 2? > But, what are these false positives? What are the consequences? Point 2 means that gapless decoding will be disabled for all streams played from HTTP. That can also hit a set of files that you put on a HTTP server as means of distribution -- where you indeed want the gapless decoding, usually. Point 3 is about doing the gapless work as normal, to indeed chop the leading jingle to its intended sample-accurate length, but then reasoning about all the MPEG-shaped data still coming and deciding that one might treat it as the next "file". This was meant regardless of the stream coming from HTTP or not (as libmpg123 doesn't really care about that) -- and so could start interpreting junk that people attached to MPEG files locally. People do everything feasible. Fact is, though, that curently, libmpg123 indeed parses (and decodes, if possible) the trailing junk, but does not produce any output samples because of the sample count limit. That needs to change anyway; gapless decoding should at least return MPG123_DONE and stop parsing. > Since good error recovery is apparently a feature of yours, recovering from > this might be good. Damn, now you cornered me! ;-) > The fact that far more people listen to net.radio than play multiple files > from http on a command line! So IMHO mpg123 should be concentrating on the > net.radio listeners more. Hm, well, I don't like trade-offs until it is clear that one cannot reach a compromise without losers... that's what this discussion is for. Alrighty then, Thomas.

Re: [mpg123-users] Questions about gapless cutting on http streams or otherwise contatenated files. From: Tom Swirly <tom@sw...> - 2011-01-11 19:09 Attachments: Message as HTML      On Tue, Jan 11, 2011 at 3:17 AM, Thomas Orgis <thomas-forum@...: > > There might be an API possibility where you return a special code to the > > calling program and allow them to either continue reading or not... > > Ah, well, it is a question how much the libmpg123 library can do. Ususally, > it's the program using the library that knows if there is web radio going on > or not. Then it can just disable gapless code. Perhaps the library could try > to mangle the default to off when it gets an ICY interval... but I'm not > sure if this logic should go there. > I agree1 > > > I don't personally understand the difference between cases 2 and 3... or > is > > 3 simply an extension of 2? > > > But, what are these false positives? What are the consequences? > > Point 2 means that gapless decoding will be disabled for all streams played > from HTTP. That can also hit a set of files that you put on a HTTP server as > means of distribution -- where you indeed want the gapless decoding, > usually. > > Point 3 is about doing the gapless work as normal, to indeed chop the > leading jingle to its intended sample-accurate length, but then reasoning > about all the MPEG-shaped data still coming and deciding that one might > treat it as the next "file". This was meant regardless of the stream coming > from HTTP or not (as libmpg123 doesn't really care about that) -- and so > could start interpreting junk that people attached to MPEG files locally. > People do everything feasible. Fact is, though, that curently, libmpg123 > indeed parses (and decodes, if possible) the trailing junk, but does not > produce any output samples because of the sample count limit. That needs to > change anyway; gapless decoding should at least return MPG123_DONE and stop > parsing. > Very clear. So you're leading us to one conclusion, really, and that's that ideally the person using the library in their code should explicitly decide whether or not any stream is gapless or not. Here's a first cut... three decoding modes. ONE_SHOT: you read one mpg file/equivalent and stop cleanly, regardless of whether there's any further data coming. GAPLESS: when one file finishes, you continue to read on that stream. BEST_GUESS: This is what older code will use by default - where you look ahead at material after the end of one mpg file, attempt to parse at least some of it, and continue the stream if successful (adjusting the sample length) or emit MPG_DONE (and a warning to stderr) if you can't parse the remaining data in the file or stream. > > > Since good error recovery is apparently a feature of yours, recovering > from > > this might be good. > > Damn, now you cornered me! ;-) > > > The fact that far more people listen to net.radio than play multiple > files > > from http on a command line! So IMHO mpg123 should be concentrating on > the > > net.radio listeners more. > > Hm, well, I don't like trade-offs until it is clear that one cannot reach a > compromise without losers... that's what this discussion is for. > > > Alrighty then, > > Thomas. > > > > ------------------------------------------------------------------------------ > Gaining the trust of online customers is vital for the success of any > company > that requires sensitive data to be transmitted over the Web. Learn how to > best implement a security strategy that keeps consumers' information secure > and instills the confidence they need to proceed with transactions. > http://p.sf.net/sfu/oracle-sfdevnl > _______________________________________________ > mpg123-users mailing list > mpg123-users@... > https://lists.sourceforge.net/lists/listinfo/mpg123-users > > -- /t http://radio.swirly.com - art music radio 24/7 366/1000

Re: [mpg123-users] Questions about gapless cutting on http streams or otherwise contatenated files. From: Thomas Orgis <thomas-forum@or...> - 2011-01-12 10:40 Attachments: signature.asc      Am Tue, 11 Jan 2011 14:09:09 -0500 schrieb Tom Swirly <tom@...>: > person using the library in their code should explicitly decide whether or > not any stream is gapless or not. Since many people seem to write apps for streaming, this should be a prominent FAQ, I guess... for endless streaming, set MPG123_GAPLESS to off. > Here's a first cut... three decoding modes. Hm, the naming confuses me a bit. > ONE_SHOT: you read one mpg file/equivalent and stop cleanly, regardless of > whether there's any further data coming. So, you mean what happens by default _now_, with gapless enabled (disregarding the still occuring needless parsing). > GAPLESS: when one file finishes, you continue to read on that stream. So ... MULTI_GAPLESS ... like, you cat together several gaplessly encoded MP3s and the decoding result will be equivalent to the concatenated outputs of the individual files... > BEST_GUESS: This is what older code will use by default - where you look > ahead at material after the end of one mpg file, attempt to parse at least > some of it, and continue the stream if successful (adjusting the sample > length) or emit MPG_DONE (and a warning to stderr) if you can't parse the > remaining data in the file or stream. Yeah, so ... that is a point: The sample length. One has to take care that mpg123 doesn't suddenly deliver more samples than it announced via mpg123_length() (after a scan). In effect, one would have to juggle with multiple gapless headers ... this could get nasty. But actually, what is the real difference between BEST_GUESS and GAPLESS/MULTI_GAPLESS? To get these modes right, one needs to invest some thinking and testing ... time. The short-term issue is to get gapless decoding deactivated where it makes obvious sense, in the mpg123 command line app. Users of libmpg123 need to be educated about disabling MGP123_GAPLESS if they do web radio. I do wonder if it's worth the effort to get a combination of gapless decoding and wildly concatenated files working. Alrighty then, Thomas.

1 message has been excluded from this view by a project administrator.