#1070 LAME 3.100 Encoded_Library string issue

[MuldeR]

Hello.

I noticed the following result from an LAME 3.100 (official release) encoded MP3 file:

<Encoded_Library dt="binary.base64">TEFNRTMuMTAwBMK5</Encoded_Library>

First of all, I'm a bit surprised that this tag's content is Base64 encoded. I haven't seen this in other tags. Not even in those that contain (Non-ASCCI) Unicode strings. Anyway, the real issue is that if I decode the Base64 data and interpret it as UFT-8, then I will get LAME3.100¹. Note that the last two characters look "scrabled", so probably that is not correct?

Best Regards,
MuldeR

[MuldeR]

Just to be sure, this was with MediaInfo v0.7.99.

In hexadecimal, theresulting string is 4c 41 4d 45 33 2e 31 30 30 04 b9.

[Jerome Martinez]

it is base64 encoded due to non-ASCII bytes due to bad library name decoding due to change in the length of the string du to number 100 :).
A GitHub ticket was just open, working on it right now with a better detection of LAME tag (and version number size).

[MuldeR]

Thanks for the quick reply!

Interestingly, the XML output genereted by MediaInfo is in UTF-8 encoding, and meta tags containing Non-ASCII charcters are just written as pure UTF-8 (no Base64 wrapping).

Encoded_Library with LAME 3.100 is the first time I have encoutered Base64 encoding in XML.

Best Regards,
MuldeR

[Jerome Martinez]

0x01 can not be in an XML (1.0) valid char, so the only possibility is to switch to an encoding.
https://en.wikipedia.org/wiki/Valid_characters_in_XML

[MuldeR]

Ah, makes sense!

[Robert Hegemann]

May I suggest to check byte $A5 Info Tag revision + VBR method from Lame info tag, instead of guessing the version number from some text?

http://gabriel.mp3-tech.org/mp3infotag.html

[Jerome Martinez]

Issue is that there is no "standard" about info tag, I don't want to try to parse a tag if I am not sure this is an Info tag (is there an agreement from all people about the idea that the content is 9 bytes of library name + version then such tag?).
also I see some issues with e.g. LAME 3.89, no Info tag ($A5 is a space so 0x20, so revision number is 2, which is not illegal, just reserved, VBR method is unknown so legal too)

[Robert Hegemann]

I know it is not perfect either. So far there is only one revision of this extra info. I don't think there will be another one in the next few years.

[Jerome Martinez]

I was not thinking there will be a version 3.100 and it is the case now ;-).
If I test only $A5, I'll discard future v2 and people will see a problem as they see today a problem with v3.100.
Trying to find a compromise.

[Jerome Martinez]

trying to avoid issues with < 3.90 (example in the link you provide: "LAME3.12 (beta 6)", I need to avoid such version because info tag is not present), thinking to do that ("Tag" is the begin of the name):

if ( Tag[4]> '3'                                    // v4 or more
 || (Tag[4]=='3' && Tag[6]=='9')                    // v3.9yz-v3.9yz
 ||  Tag[4]=='3' && Tag[8]>='0' && Tag[8]<='9')     // v3.xy0-v3.xy9

[Robert Hegemann]

The text could be anything, there is no garantee. If at $A5 is a character in range 0x10 to 0x1f, then it can't be a revision 0 tag. And as you already noticed, it can only be a revision 1 tag. If we want to invent another revision, which has to be backward compatible with previous revisions anyway, then we will have to implement something that is not just by increasing some value at $A5. Maybe we would code it at $A6?

[Jerome Martinez]

The text could be anything, there is no garantee.

Issues here ares:
- Versions of LAME <3.90 do not have this info tag
- Any value of $A5 is valid (same rule "could be anything, there is no garantee" is applied, you choose to prioritize $A5 over version number and I choose to prioritize version number over $A5, none is more legitimate then the other and I choose the version number in htat case)
Example: "LAME3.89 (" is valid with current specs (and your sentence), but has no Info tags and library string size is not 9, so I need to filter it in some way.

So either the "spec" changes and says that any value of $A5 >=0x20 is invalid, or we can not rely on $A5<0x20 (e.g. "LAME3.101\x20" or "LAME3.101(" are valid with spec, not with your suggestion)

Your suggestion "byte $A5 Info Tag revision + VBR method from Lame info tag" does not work here, as the spec says that all values are valid ("reserved" or "unknown", that means valid for me), I don't see how to rely on it more than the LAME version number.

For the moment, I see that I need to filter the LAME version number because LAME <3.90 have a different schema and the "(" is valid for version+VBR info (just "unknown") if I don't test the version.

I am open to suggestions, but they must be better than my arbitrary choice (not just "equal" about how they are legitimate).
Reason behind my choice: if I don't recognize the version number, I display the 20 chars as is and is easy to see that the firsts bytes of the library so what is the new version (also if more than 9 chars are chosen by the encoder)

Please suggest a test (and/or specs chages?) which handles both LAME <3.90 and LAME >=3.90 as well as not discarding valid values of other bytes.

PS: About backward compatibility, the spec is also not clear: "A few fields, as they could be necessary for some functionnalities of already existing software, should not be moved in any version of the tag. They are indicated as "UNMOVABLE". " but nothing is "UNMOVABLE" so why this sentence?
And worse, it is "rev" word which is usually used in specs for backward compatible update (in opposition to "ver" which as actually sometimes also backward compatible), so IMO it is not clear that revisions are backward compatible or not.
IMO the spec should be updated and being clear about if revisions are backward compatible (e.g. "decoders MUST ignore any rev >1" tag) and how to detect old method (20-char strings) and new method (9-char string + info tag) because currently there is not explicit rule about how to do the difference, the spec is not clear enough about that, so software must imagine rules (and each software will decides differently)

[Jerome Martinez]

Fixed, with Windows snapshot available.
Note: this is an hot fix, there is a debate about how to detect new LAME versions and corresponding Info tag, discussion is not closed in case a better method for detecting new versions is found.

[MuldeR]

Thanks! Looks good now for the files I have tested.

[lazka]

In case you need more test files: https://github.com/lazka/mutagen-mp3-test-vector

[Jerome Martinez]

"8,992 entries", ouch :)
Interesting, thanks!
Now, I need to find the time for testing them and see if MediaInfo uoutput is fine enough...

[MuldeR]

Interesting stuff.