#384 The UFID frame of an ID3v2 tag does not get parsed

[Remilia Scarlet]

It looks like libmpg123 (and by extension, mpg123-id3dump) doesn't recognize the UFID frame of an ID3v2 tag. I checked this against both an ID3 v2.4.0 and an ID3 v2.3.0 tag. The file I used to check this with for sure had a UFID tag, as the id3v2 program displayed it just fine.

[Thomas Orgis]

1. Sure, mpg123's ID3 parsing is not exhaustive. It is mostly about (text) fields useful for player display.
2. Can you give an example file and also show the id3v2 output?

Seems like id3v2 doesn't actually create UFID frames.

id3v2 --UFID xxx foo.mp3

does not result in anything being added to the file. So I wonder how you populate these and also, since the data is supposed to be unspecific binary, how it is to be displayed.

This field seems to be designed for the same purpose that Amazon utilizes a simple comment for. Who is employing UFID?

Bonus question: Which other ID3 fields that mpg123 does not handle do you consider relevant?

[Remilia Scarlet]

So I came across this while adding support for MusicBrainz IDs to the music player I wrote, Benben, which uses libmpg123 for MPEG-1 and ID3 support. Support for these IDs was being added because, when you submit to ListenBrainz, adding MusicBrainz IDs (if they exist in the file) help ensure a better match for the user. And it turns out, MusicBrainz Picard uses the UFID frame to store the MusicBrainz Recording ID.

I'm not totally sure what other fields aren't getting handled, but I can spend some time to explore that if you'd like.

I'm attaching an example file that had a UFID tag added with Picard (it's my own work, all good). Here's the output from id3v2:

[alexa@flandre ~]$ id3v2 -l 13\ -\ Colorado.mp3
id3v1 tag info for 13 - Colorado.mp3:
Title  : Colorado                        Artist: Partition 36
Album  : Waveforms                       Year: 2014, Genre: Electronic (52)
Comment:                                 Track: 13
id3v2 tag info for 13 - Colorado.mp3:
TIT2 (Title/songname/content description): Colorado
TPE1 (Lead performer(s)/Soloist(s)): Partition 36
TRCK (Track number/Position in set): 13/13
TALB (Album/Movie/Show title): Waveforms
TPOS (Part of a set): 1/1
TXXX (User defined text information): (SCRIPT): Latn
TPE2 (Band/orchestra/accompaniment): Partition 36
TSO2 ():  frame
TSOP ():  frame
TMED (Media type): Digital Media
TXXX (User defined text information): (originalyear): 2014
TCOM (Composer): Alexa Jones-Gonzales
TXXX (User defined text information): (ARTISTS): Partition 36
TXXX (User defined text information): (CATALOGNUMBER): PART-005
WOAR (Official artist/performer webpage): http://www.partition36.com
WXXX (User defined URL link): (): http://www.partition36.com
TXXX (User defined text information): (MusicBrainz Album Type): album
TXXX (User defined text information): (MusicBrainz Album Status): official
TXXX (User defined text information): (MusicBrainz Album Release Country): XW
TCOP (Copyright message): Copyright (C) 2014 Alexa Jones-Gonzales
TXXX (User defined text information): (MusicBrainz Album Id): b43d606b-9f52-44a9-8ccf-587a617c5e24
UFID (Unique file identifier): http://musicbrainz.org, 36 bytes
TXXX (User defined text information): (MusicBrainz Artist Id): 02754aa4-b64a-4287-ac87-e20e3b01d28d
COMM (Comments): ()[XXX]: Creative Commons 3.0 Attribute-NonCommercial-ShareAlike
TXXX (User defined text information): (MusicBrainz Album Artist Id): 02754aa4-b64a-4287-ac87-e20e3b01d28d
TXXX (User defined text information): (MusicBrainz Release Group Id): 4ecb89f1-754e-480c-a384-dbf4e28b16d7
TXXX (User defined text information): (MusicBrainz Release Track Id): 50822202-0ac7-416f-bcd9-4f76bcb1fb84
APIC (Attached picture): ()[, 3]: image/jpeg, 46712 bytes
TORY (Original release year): 2014
TYER (Year): 2014
TDAT (Date): 2504
TCON (Content type): Electronic (52)
[alexa@flandre ~]$

[Thomas Orgis]

Hm, so id3v2 is not actually displaying the binary data for UFID. I thought I'd let mpg123-id3dump show it in hex.

Regarding additional fields: I guess you'd like to see the URL fields (starting with W), too. Then there's TORY. There is a numer that I did not cover. Adding more text-like fields is easy. The binary ID needs some specific bits, but not such a big deal.

So far there was no demand for more fields, but sure, I can add more.

[Remilia Scarlet]

Awesome, sounds good.

[Thomas Orgis]

Can I keep the leading bits of your file as test data in the mpg123 distro?

I guess I'll just add parsing and testing of the dump for the fields it has.

[Thomas Orgis]

Revisiting the ID3v2 spec … there are lots of frames that I never encountered, ideas for specifc uses that did not materialize. As I said, adding text fields should be easy enough, but there is lots of funny binary frames, or things like LINK … I do not think that it is libmpg123's task to provide a full implementation of the standard.
It is unfortunate that you got lots of MusicBrainz IDs stored as TXXX frames and only one of them as UFID.

I am pondering how to add API for the UFID frames. It needs a different type than the texts, so either an extension of the mpg123_id3v2 struct or a separate API call.

I guess the separate API call is the safe way to ensure that a client program encountering an older libmpg123 dies in a controlled way. But somehow I'd like to keep down the count of symbols in the library. Maybe I can do something with a flag setting / feature query.

[Remilia Scarlet]

Sorry for the late reply! Yeah, keep the whole file if you wish, it's fine with me.

I don't think support for every single frame would be necessary if it adds additional symbols to the library. Just ones with substantial use in the wild and strong use cases.

[Thomas Orgis]

So this is still on the list. I'll get around to it eventually.

[Remilia Scarlet]

Sure thing, no rush on my end.

[Thomas Orgis]

Say, is there anything that speaks against just treating UFID frames like text frames? It doesn't have a language, but the owner/URL maps to a description and the content can just be a hex string for the up to 64 bytes.

So mpg123-id3dump would print

UFID description(http://musicbrainz.org)
 E7C22B994C59D9CF2B48E549B1E24666636045930D3DA7C1ACB299D1C3B7F931F94AAE41ED

for a 36 byte identifier. The information would be stored, hex-encoded as text in either in mpg123_id3v2.text or mpg123_id3v2.extra. I'm pondering which. I guess it should be extra. This would also match things like MCDI. So this small-ish binary stuff just formatted as text and squeezed in there. The URL fields can also easily fit.

All this protected by an opt-in flag MPG123_ID3V2_MORE or similar so that old programs are not surprised to find something other than TXXX in the extra list.

Would that work for you?

[Remilia Scarlet]

Yeah, that approach makes sense to me, and would work perfectly for my specific case. It's kind of a strange field because it has both text and binary data in it.

[Thomas Orgis]

OK, I'll get into that then somewhen. About the placement: texts is for unique ID3 frame IDs, extra is for multiple TXXX that are distinguished by description. So that fits UFID, of which there could be many. The W URL frames should go into texts then, except for WXXX.

Anything that is unique without description goes into the text array.

I'll only have to think about hiding the additions behind an opt-in flag or not. I think clients should not be surprised by added text entries, but they might be surprised at a non-TXXX appearing in extra.

[Thomas Orgis]

I finally added this for UFID. Other extra text fields should follow for mpg123 1.34.0. For your test file, mpg123-id3dump output now includes:

UFID description(http://musicbrainz.org)
 61363061343237322D383734642D346131372D613532372D333663646364313861643136

This is added to mpg123_id3v2.extra as discussed. I think we don't need a switch for this.

Can you verify that this makes sense for your application, and that the rendering is correct?

Edit: Of course I'm talking about current svn trunk wich is reflected in https://mpg123.org/snapshot in a few minutes.

[Remilia Scarlet]

Works perfectly! I grabbed the code, made a Slackware package on one of my machines to replace the system one temporarily, then did a quick test with the Lisp version of my code in a REPL:

CL-USER> (asdf:load-system :cl-remiaudio-codecs-mpeg1)
T
CL-USER> (let* ((dec (cl-ra/mpeg1:make-mpeg1-decoder #P"/home/remilia/tmp/benben-test-files/01 - Noise Marines.mp3"))
                (bundle (find "UFID" (cl-ra/mpeg1:id3-extras (cl-ra/mpeg1:decoder-id3 dec))
                              :test #'string=
                              :key 'cl-ra/mpeg1:text-bundle-id)))
           (when bundle
             (print (cl-ra/mpeg1:text-bundle-description bundle))
             (let ((str (cl-ra/mpeg1:text-bundle-text bundle)))
               (loop for idx from 0 below (length str) by 2
                     for substr = (subseq str idx (+ idx 2))
                     collect (parse-integer substr :radix 16) into numbers
                     finally (return (sdm:bytes->string numbers))))))

"http://musicbrainz.org" 
"fd46ee3f-ceae-4e8e-b817-2f171393511d"

That ID is for https://musicbrainz.org/recording/fd46ee3f-ceae-4e8e-b817-2f171393511d and that's exactly what I expected to see. Thank you so much!! :D