Menu

#97 Strips all tags when processing too many files

3.1.2
closed
nobody
None
1
2019-04-26
2014-09-29
No

Hi Urs. I've been going through my collection filtering for mp3 files with v2 & v1 tags, expanding the folders, selecting the files, then removing the v1 tag. If I select too many file at once, it does a full churn on one CPU (normal) but stops after a seemingly pre-set amount of time; any files after a certain number say "NO TAG", so I close kid3 to prevent them from saving. I've been doing them in small batches because of this, but I thought I'd register the bug anyway.

Cheers

Simon

Related

Bugs: #97

Discussion

  • Urs Fleisch

    Urs Fleisch - 2014-10-01

    Thanks for the report. I can reproduce similar things if I perform file changing operations on a lot of files which are using TagLib (e.g. MP3 files with ID3v2.4.0 tag or by disabling id3lib). The problem is that TagLib keeps an open file descriptor for every audiofile object. This made problems on Mac OS X, because it has a rather low maximum of open file descriptors. So I fixed this for [bugs:#65] by keeping a limited number of files open. However, I have to keep the file descriptors of modified files open, because the changes are stored in the TagLib audiofile object. So you still have this problem if you have many modified files.

    Later versions of TagLib introduced a new IOStream abstraction, I will have a look at this, if it can be used to really solve the problem. In the mean time, you will have to keep the number of concurrently modified files below the count of maximum file descriptors. As a workaround, you could try to increase file descriptor maximum. See for example http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/, which mentions ulimit -Hn, ulimit -Sn, /proc/sys/fs/file-max or http://askubuntu.com/questions/162229/how-do-i-increase-the-open-files-limit-for-a-non-root-user.

     

    Related

    Bugs: #65

  • Urs Fleisch

    Urs Fleisch - 2014-11-04

    I have reduced the usage of file descriptors in situations with a lot of modified files. You can find binaries in http://sourceforge.net/projects/kid3/files/kid3/development/ or you can build the code from Git. Could you please tell me if it solves this problem? There is also a fix which should solve the other problems you reported, so could you please check if this problems are still present with this development version?

     
    • Simon Dedman

      Simon Dedman - 2014-11-04

      Nice work, cheers Urs! I'm afraid I won't be able to check for a month or
      so as I'm away from my Linux machine until early December but I'll put it
      on the to do list! :)
      On 4 Nov 2014 17:47, "Urs Fleisch" ufleisch@users.sf.net wrote:

      I have reduced the usage of file descriptors in situations with a lot of
      modified files. You can find binaries in
      http://sourceforge.net/projects/kid3/files/kid3/development/ or you can
      build the code from Git. Could you please tell me if it solves this
      problem? There is also a fix which should solve the other problems you
      reported, so could you please check if this problems are still present with
      this development version?


      Status: open
      Group: 3.1.2
      Created: Mon Sep 29, 2014 06:12 PM UTC by Simon Dedman
      Last Updated: Wed Oct 01, 2014 04:49 AM UTC
      Owner: nobody

      Hi Urs. I've been going through my collection filtering for mp3 files with
      v2 & v1 tags, expanding the folders, selecting the files, then removing the
      v1 tag. If I select too many file at once, it does a full churn on one CPU
      (normal) but stops after a seemingly pre-set amount of time; any files
      after a certain number say "NO TAG", so I close kid3 to prevent them from
      saving. I've been doing them in small batches because of this, but I
      thought I'd register the bug anyway.

      Cheers

      Simon

      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/kid3/bugs/97/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

       

      Related

      Bugs: #97

  • Urs Fleisch

    Urs Fleisch - 2014-11-09
    • status: open --> closed
     
  • Urs Fleisch

    Urs Fleisch - 2014-11-09

    Fixed in version 3.1.2.

     
  • Simon Dedman

    Simon Dedman - 2019-04-23

    Hi Urs,
    It's an old bug, but it checks out - just had this happen again with 3.6, upgraded to 3.7latest and it's still happening:
    Extracted zip file of mp3 album just bought, 'Name From Tag' for whole folder, works for many, strips tags from 2. Previously it stripped from more than that so it doesn't seem consistent. I can well believe your explanation above is correct; would it be possible to workaround by limiting concurrency? I'm sure this would make it slower but surely that wold be better than it failing? Let me know if you want me to send you the zip for confrmation, though I suspect there's nothing special about these MP3s.
    Cheers pal,
    Simon

     
  • Urs Fleisch

    Urs Fleisch - 2019-04-24

    Hi Simon,
    Unfortunately, I have never experienced such a behavior, but to fix it, I will have to reproduce it. So let me make sure that I understand the problem correctly: You have a folder containing some files having ID3 tags, you select them all, their icons show that they have tags (e.g. TAG 2). Now you click the "From Tag" button to generate file names from the tags. After that, some of the files show "NO TAG" (and obviously the file name was not generated for such files). Is this correct? What operating system are you using? Could you provide me access to such a zip file via a private message?
    By the way: All these operations run in a single thread, there is no concurrency. I thought that the limited number of open file descriptors was the problem, but as you describe, these are only files from a single album, so they cannot be so many...
    Regards,
    Urs

     
  • Simon Dedman

    Simon Dedman - 2019-04-24

    Hiya,
    Your summary is correct yeah.
    picture
    Files
    I'm on quite an old machine this might not reproduce on something with a lot more resources but let me know how you get on and I'll see if I can help.
    Cheers!
    Simon

     
  • Urs Fleisch

    Urs Fleisch - 2019-04-26

    Hi Simon,

    Although my machine is old too (Core2Duo), I am still not able to reproduce any problem. According to your screenshots, I assume that you are using kid3-qt on Linux and that the files you are processing are not on a network share. Could you please try the following procedure to check if the problems are visible at the level of system calls.

    • Make sure that you have strace and pgrep installed
    • Start kid3-qt, proceed just before the errors start to happen
    • Now in a terminal, start the following command:
      strace -p $(pgrep kid3-qt) 2>strace.log
    • Proceed with Kid3 until the bug occurs, then close Kid3

    You can send me the resulting strace.log (compressed as it will be quite large) or have a look at it yourself to see if there is something suspicious.

    Best regards,
    Urs