#67 MKV with large number of streams stops file scanning

open
None
5
2011-04-20
2011-04-18
Anonymous
No

I'm not sure what the limitations are, but I had an MKV file with 20 streams (1 video, 3 audio, 16 subtitle tracks) that would stop the file scanner. Everything found before that file would be available via the program but anything after (and including that particular file) would not be served.

I remuxed the file using mkvtoolnix, keeping all the streams the same, and still had problems. I remuxed the file keeping the video, audio and only ONE subtitle track and the program resumed normal operation. I don't have time, right now, to figure out how many streams is too many but I thought that I would let everyone know.

It is only a minor bug so I don't expect it to get fixed anytime soon.

Discussion

  • Justin Maggard
    Justin Maggard
    2011-04-20

    Any way I can get a copy of the file for debugging purposes? I can't think of a way that MiniDLNA itself would have problems with that many streams, but perhaps libavformat can't handle it though.

     
  • Justin Maggard
    Justin Maggard
    2011-04-20

    • assigned_to: nobody --> jmaggard
     
  • Sorry for not checking back sooner.

    I've tried several different types of MKV files, all with the same result. The common theme with all is a large number of streams. I've also tried to remux them with mkvmerge (v3.0.0) to see if the original container were somehow broken.

    As far as libavformat goes, I've been able to play the files (and streams) successfully with Totem (v2.30.2) (I believe that it uses libavformat). The version that I have is 0.5.1. VLC Player also has no problems but I realize that it uses its own decoding libraries (just added here for additional information).

    As far as getting you a copy of the files that I'm having trouble with; they would be problematic to send because of their large size.

    I hope this helps and I will try to check back sooner.

     
  • Justin Maggard
    Justin Maggard
    2011-05-23

    I found a sample MKV file at https://roundup.libav.org/file919/sample.mkv that contains 28 streams (1 audio, 1 video, 26 subtitles), and it doesn't work with libavformat, but it also doesn't crash minidlna. Maybe you can try it? From the info I found at https://roundup.libav.org/issue963 though, ffmpeg is limited to a max of 20 streams. So there doesn't appear to be anything I can do here.

     
  • jmaggard, thank you for your patience through all of this.

    I believe that I've mixed 2 separate issues/bugs together and I have used incorrect/unspecific terminology to explain what I am experiencing.

    Firstly, I now, correctly, remember that I was/am experiencing a 'crash' of miniDLNA on large stream files on my D-Link DNS-323 on which I have installed miniDLNA. (That version was compiled specifically for the DNS-323). In that case ONLY, the program would simply stop scanning after it encountered a file with a large amount of streams. So, if there were 10 files and the 2nd one alphabetically had a large amount of streams, it would only list the 1st file.

    I cannot be precise on exactly how many streams there were in the files that I experienced this with but probably in the ballpark of 20 (maybe less, maybe more). This is an older version of miniDLNA (v1.0.18). In this case, I'm guessing a newer version probably doesn't exhibit this exact problem.

    On my Ubuntu PC (running miniDLNA), I have found that files with a large number of streams have simply not shown up in the list of available media. BUT, it has not (as I incorrectly described) 'crashed'. It, in fact, continued scanning the directory finding other files and adding them to the database.

    I can confirm that the sample you suggested doesn't work via miniDLNA but I have found a couple of files that also don't work that I am using that have less than the 20 stream limitation. One, in fact, has 17 streams (1 video, 3 audio, 13 subtitles). I no longer have the other files because I was freeing up space on my HDD and simply deleted them.

    I was, however, able to access all the streams from the sample.mkv file via Totem (the version I mentioned below). I'm not sure if that means that libavformat is working or if Totem has a work-around or simply doesn't use it.

    I am sorry that I mixed these 2 issues together and thus created a large headache for you (I'm sure) and I appologize for that. I believe you can ignore the issue that I was/am having related to my DNS-323 as it is an old version of miniDLNA.

    When I have a little more free time (and disk space), I will try to create multiple files with differing stream numbers to determine just how many is too many and report back to you (here). Perhaps there is something specific with the files themselves that I am using that is the problem. Seeing as I am probably the only one with this problem, I believe you can downgrade the priority of the bug. No need to waste your time if I'm the only one seeing this issue (if it does, in fact, have anything to do with miniDLNA).

    Again, thanks for your help and time.

     

  • Anonymous
    2011-09-21

    I can confirm that!

    The scanner stops directly before the mkv wihtout any error-message.

    mDLNA 1.0.22

     
  • Robin Kluth
    Robin Kluth
    2011-11-07

    Hey there.

    Just build mdlna from csv-head and I try to let mdlna create its database.
    Two files are skipped:

    ============/var/log/minidlna.log============
    [2011/11/07 21:31:21] minidlna.c:903: warn: Starting MiniDLNA version 1.0.22 [SQLite 3.6.22].
    [2011/11/07 21:31:21] minidlna.c:926: warn: Creating new database at /var/cache/minidlna/files.db
    [2011/11/07 21:31:21] minidlna.c:996: warn: HTTP listening on port 8200
    [2011/11/07 21:31:21] scanner.c:733: warn: Scanning /home/samba/shares/filme
    [2011/11/07 21:31:29] metadata.c:720: warn: Opening /home/samba/shares/filme/Just Go With It.mkv failed!
    [2011/11/07 21:31:29] scanner.c:506: warn: Unsuccessful getting details for /home/samba/shares/filme/Just Go With It.mkv!
    [2011/11/07 21:31:35] metadata.c:720: warn: Opening /home/samba/shares/filme/Rio.mkv failed!
    [2011/11/07 21:31:35] scanner.c:506: warn: Unsuccessful getting details for /home/samba/shares/filme/Rio.mkv!
    [2011/11/07 21:31:40] scanner.c:804: warn: Scanning /home/samba/shares/filme finished (183 files)!
    [2011/11/07 21:31:40] playlist.c:100: warn: Parsing playlists...
    [2011/11/07 21:31:40] playlist.c:223: warn: Finished parsing playlists.
    ============/var/log/minidlna.log============

    Do you need further file-info's?

    OS: Ubuntu 10.04.3
    libavformat52

    The two files are not shown over my UE32C6200.

    I hope this problem can be fixed ;)

    Thanks for your great tool!

     
  • Justin Maggard
    Justin Maggard
    2011-11-09

    Looks like a busted libavformat to me. Can you post the output of "ffprobe -pretty -show_streams"?

     
  • Robin Kluth
    Robin Kluth
    2011-11-09

    Thanks for your answer!

    ====================================
    FFprobe version SVN-r20090707, Copyright (c) 2007-2009 Stefano Sabatini
    libavutil 49.15. 0 / 49.15. 0
    libavcodec 52.20. 0 / 52.20. 1
    libavformat 52.31. 0 / 52.31. 0
    built on Jan 19 2010 21:59:04, gcc: 4.4.3 20100116 (prerelease)
    You have to specify at least one input file.
    ====================================

    with input file:

    ====================================
    FFprobe version SVN-r20090707, Copyright (c) 2007-2009 Stefano Sabatini
    libavutil 49.15. 0 / 49.15. 0
    libavcodec 52.20. 0 / 52.20. 1
    libavformat 52.31. 0 / 52.31. 0
    built on Jan 19 2010 21:59:04, gcc: 4.4.3 20100116 (prerelease)
    [matroska @ 0x90f9120]Unknown/unsupported CodecID S_HDMV/PGS.
    Just Go With It.mkv: memory allocation error occurred
    ====================================

    Many thanks ;)

     
  • Justin Maggard
    Justin Maggard
    2011-11-09

    Okay, so indeed the problem is in libav*, and outside the control of minidlna. You may want to try using the Ubuntu 11.10 packages, or compiling the ffmpeg libs yourself from the latest upstream version.

     
  • Robin Kluth
    Robin Kluth
    2011-12-20

    Thanks for your answer.

    I've had to set-up the server again, but libavformat or any other lib used by mdlna are not installed, but the stazic-package works.

    Which libs are used? Or has the static-package all files need to running?

     
  • Justin Maggard
    Justin Maggard
    2011-12-21

    That static package doesn't have any external dependencies. If your server is running Ubuntu 11.10 now, you can also just install the Ubuntu package with apt-get.

     
  • Robin Kluth
    Robin Kluth
    2011-12-21

    No, just set-up Zentyal 2.2 again.

    If the static has all dependencies inside, could you build a new one with newer versions of the ffmpeg libs?

    Zentyal only supports LTS-releases of Ubuntu, so the next Ubuntu-upgrade is April next year.

     
  • Robin Kluth
    Robin Kluth
    2012-01-02

    Happy new year @ all :)

    @ jmaggard: No update on this issue, yet - Is it possible that you can build a new static-version?
    How can I build one? The source-code (Makefile) does not have any comments about this.

    Thanks ;)

     
  • Robin Kluth
    Robin Kluth
    2012-02-17

    Hi!

    Just testing the newest static-package of mindiDLNA - but it doesnt work for me jmaggard :(

    It hangs on this mkv: http://pastebin.com/5D8mhevR

    What makes miniDLNA stop on this file?

     
  • Robin Kluth
    Robin Kluth
    2012-02-18

    Thanks for this link jmaggard!

    It doesnt work for me (replaced the old minidlna binary with this one), same error: It hangs on:

    [2012/02/18 18:18:48] metadata.c:853: debug: Container: 'matroska' [BLITZ - CopKiller vs. KillerCop.mkv]

    Some more infos about the file:

    -rwxrwxrwx+ 1 root __USERS__ 25907396655 2011-12-09 16:46 /home/samba/shares/filme/BLITZ - CopKiller vs. KillerCop.mkv

    Is this binary static-linked? If not, it using the old versions from my system.

    Hope I can help you with my informations ;)

     
  • Robin Kluth
    Robin Kluth
    2012-02-27

    Hi again,

    did you sime research for this issue? If not (and you have time to test it), please set up a Ubuntu 10.04.3 Server and try your build with an similar mkv file.

    If you need further informations, post a message ;)

     
  • Robin Kluth
    Robin Kluth
    2012-04-13

    Something new on this ticket?

    Latest static package does not work. Again, the first big mkv with a number of streams make mDLNA to hang on this file.

    Its a static-build, so the libav* bug is still there. Exists a newer version of libv* to try?

     
  • Justin Maggard
    Justin Maggard
    2012-04-23

    No, nothing new. Again, this issue lies outside of minidlna, and I can't really do much without having a file that can reproduce it.

     
  • Robin Kluth
    Robin Kluth
    2012-10-26

    Hi, again.

    Just a status update: Iam now on Ubuntu 12.04.01 LTS (Zentyal 3.0) with miniDLNA 1.0.25-static.

    File scanning still stops at following file (at least minidlna stops on this line in debug mode. it showing:

    =============
    [2012/10/26 14:37:55] metadata.c:883: debug: Container: 'matroska' [Batman - Year One.mkv]
    =============

    File information: http://img.xrmb2.net/images/563573.png

    I'd really like if you could check that file, I'll mail you again.

     
  • Robin Kluth
    Robin Kluth
    2012-10-26

    Sorry for the spam.

    I think I've found the file which causes this: A 18GB .mkv with 21 Streams. I've put it in a extra folder and started minidlna with -d:

    minidlna does not show that file in the logs, so I assume it hangs.

    Again: I'd really want to send you this file - If that would help you!

     
  • Robin Kluth
    Robin Kluth
    2012-11-15

    Ok, got it:

    I've build minidlna from latest source with latest dev's (as of Ubuntu 12.04.1) and its working! All files are scanned!

    Isnt that easy...