#189 Set command not working with some aif files

[Teresa Soleau]

Hello, I am using kid3-cli to add descriptive metadata to a bunch of aif, mp3 and mp4 files. I'm using the following command.

kid3-cli -c "set comment 'Part of: The Kitchen videos and records (accn. no. 2014.M.6)'" -c "set title $filename" $a

Where $filename is being set earlier in the script and $a is the variable used for the file that's being edited.

For some of the aif files the data is being added (see 1st exiftool output below) but for others the data is not being added (see 2nd exiftool output below). Can you see any reason why it's working for some files and not others? Thanks for any help.

~Teresa

This is how one looks when it works (see the Comment and Title at the end):

ExifToolVersion = 11.01
FileName = gri_2014_m_6_K2000091_10_pm.aif
Directory = .
FileSize = 73184
FileModifyDate = 1544485302
FileAccessDate = 1544485302
FileInodeChangeDate = 1544485302
FilePermissions = 33272
FileType = AIFF
FileTypeExtension = AIFF
MIMEType = audio/x-aiff
AIFF 'COMM' chunk (18 bytes of data):
Common (SubDirectory) -->

• [BinaryData directory, 18 bytes]
  | NumChannels = 1
  | NumSampleFrames = 23722
  | SampleSize = 24
  | SampleRate = 44100
  AIFF 'minf' chunk (16 bytes of data):
  AIFF 'elmo' chunk (426 bytes of data):
  AIFF 'SSND' chunk (71174 bytes of data):
  AIFF 'regn' chunk (92 bytes of data):
  AIFF 'ovwf' chunk (214 bytes of data):
  AIFF 'umid' chunk (24 bytes of data):
  AIFF 'ID3 ' chunk (1144 bytes of data):
  ID3 (SubDirectory) -->
  | ID3Size = 1144
  ID3v2.4.0:
  | + [ID3v2_4 directory, 1134 bytes]
  | | Comment = Part of: The Kitchen videos and records (accn. no. 2014.M.6)
  | | Title = gri_2014_m_6_K2000091_10

This is how one looks when it doesn't work:

ExifToolVersion = 11.01

FileName = gri_2014_m_6_K2000095_05_pm.aif
Directory = .
FileSize = 248104768
FileModifyDate = 1544562210
FileAccessDate = 1544562286
FileInodeChangeDate = 1544562210
FilePermissions = 33272
FileType = AIFF
FileTypeExtension = AIFF
MIMEType = audio/x-aiff
AIFF 'COMM' chunk (18 bytes of data):
Common (SubDirectory) -->

• [BinaryData directory, 18 bytes]
  | NumChannels = 1
  | NumSampleFrames = 82486553
  | SampleSize = 24
  | SampleRate = 44100
  AIFF 'minf' chunk (16 bytes of data):
  AIFF 'elmo' chunk (426 bytes of data):
  AIFF 'SSND' chunk (247459667 bytes of data):
  AIFF 'regn' chunk (92 bytes of data):
  AIFF 'ovwf' chunk (644456 bytes of data):
  AIFF 'umid' chunk (24 bytes of data):

[Urs Fleisch]

Thanks for the report. I tried to reproduce the problem by putting all of my AIFF files (which are only a few) into a folder and then starting the following command (exactly your command, except that I put quotes around $filename and $a to cope with white space, if you have whitespace, you should do that too):

for a in *.aif*; do
    filename=${a%.aif*}
    kid3-cli -c "set comment 'Part of: The Kitchen videos and records (accn. no. 2014.M.6)'" -c "set title '$filename'" "$a"
done

After this command, all the files had the correct title and comment, i.e. I cannot reproduce the bug with my existing files. Could you please send me a file or a link to a file which shows the problem, so I can check what is going wrong?

[Teresa Soleau]

I sent you links to the files in a private message. Please let me know if you don't receive them.

[Urs Fleisch]

Thanks for the files. I have downloaded and analyzed them, so you could delete them now.

When I start Kid3 from a terminal and open the file which causes problems, the following error message is displayed:

TagLib: RIFF::File::read() -- Chunk 'reg' has invalid ID

This message comes from TagLib, which parses the AIFF files. In the TagLib sources, in riffutils.h, there is the following check

  inline bool isValidChunkName(const ByteVector &name)
  {
    if(name.size() != 4)
      return false;

All RIFF chunks with an ID which has not 4 characters are considered invalid, and probably TagLib is right, at least I did not find a document listing a chunk ID which does not have 4 characters. I open the file with the Windows RIFFPad tool, it stops at a "?reg" chunk and reports an unknown error. On the other hand "exiftool -v filename.aif" does not report any errors and displays a "regn" chunk. I will have to dive a little deeper to check if TagLib is correct.

[Teresa Soleau]

Thanks so much for looking into this. I look forward to your recommendation for how to fix the issue.

[Urs Fleisch]

I analyzed the situation further and posted a bug report and a pull request for TagLib. You can find a development snapshot, which uses a patched TagLib, in the development folder. Please tell me if it works.

[Teresa Soleau]

Great! Glad to help find a bug. :)

Unfortunately, I don't have access to install this development snapshot on our server. If you think it will be some time before it's incorporated into the code I will ask our server team to install it, but otherwise I will wait until the new version is released to have that installed.

[Urs Fleisch]

I plan to release version 3.7.0 before Christmas. But if you are using Linux, you should know that "normal" Kid3 application packages (e.g. deb, rpm) will use the TagLib version which is installed on the system, and this TagLib version will not have the patch. This will be the case when the current behavior is considered a bug, the pull request is accepted and merged and a new TagLib version is released. This could take some time - TagLib has seen higher activity than today, the last release is from October 24, 2016.

With the current development snapshot and the coming official release, you have the option to use the binary Linux package, which is provided in a tgz archive and contains a directory with the kid3-qt and kid3-cli binaries and most of the dependent libraries statically linked to the plugins - also a patched version of TagLib. You can extract this archive anywhere and execute the binaries, there is no need for administrative access. For convenience, you could add symlinks to the binaries in a directory which is in your PATH.

[Teresa Soleau]

The files I'm processing are on a server (RHEL7 I believe) and there are so many of them that I won't be bringing them locally onto my Windows machine to embed the metadata - I'll be doing it right on the server. So it sounds like I should ask my server admin to install the dev version with the patched version of TagLib. Thanks!

[Urs Fleisch]

Version 3.7.0 is now released, so your server admin can install kid3-3.7.0-Linux.tgz.

[Teresa Soleau]

I had 80 files that wouldn't embed so I reran those 80 files with this change (kid3-git20181219) and it worked for 50 of the files but there are still 30 more that won't embed. I can send you a link to a couple of the files that are still not working in a private message. Can you take a look?

[Urs Fleisch]

My TagLib patch was trying to keep the old behavior in cases where it did not lead to an invalid chunk name, but this was not a good idea. I have now fixed it to be more strict and always accept the padding if it does not directly lead to a wrong chunk name. Please have a look at version kid3-git20190210, which can be found in the development folder.

[Teresa Soleau]

Thank you! I will have this installed here and see if it works for these remaining files.

[Teresa Soleau]

I have installed and tried this out and it worked on 25 of the remaining 30 files but there are still 5 files that won't embed! So close! Can you take another look. At least this is making the tool more robust? I'll send you the remaining 5 files.

[Urs Fleisch]

Thanks for the files. I had a look at the first and the last of them, sorry for not looking at all but they are huge (>1GB) and looking at their names and sizes they seem to come from the same source and probably show the same problems. Indeed, what I have seen for the two files I inspected is the same: They start with a COMM chunk followed by an SSND chunk, but the size of the SSND chunk is wrong. When looking at the file contents near the offset pointed to by the size of the SSND chunk there seems to be nowhere near anything which could be the fourCC name of the next chunk (e.g. four consecutive characters). So I suppose the files are corrupt. Maybe the SSND chunk goes until the end of the file and you could fix it by modifying the chunk size so that it is the file size minus the offset of the first chunk byte (the size would be in the four bytes after SSND near the beginning of the file, it is big endian encoded). Maybe you can find a program which can fix this automatically. I tried RIFFpad on Windows to look at the file and tried to convert the file using sox and audacity and play it using audacious, but all these applications give only access to a few minutes of sound (probably corresponding to the wrong size) and not the duration of over an hour which is reported by mediainfo.

[Teresa Soleau]

Thank you so much for this analysis! Even if it doesn't help us embed the file it's helpful to know that it's corrupt. With so many files in this project we couldn't possibly watch/listen to all of them and with the original analog media stored in boxes it's hard to know how long they are supposed to be. This info gives us some indication that we will need to reformat them from the analog at some point. Anyway, thanks again for everything!