Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#811 Problems with MKVs after being processed with Header Editors

Crash
open
nobody
None
5
2014-04-21
2013-12-31
Chad Greenfield
No

I'll try this again,

I'm ripping my Blu Rays with MakeMKV then I use MKVPropEdit to set the Track Names and Flags and MetaX to set Meta Data. After that I have a batch file that uses MediaInfo data to create a command line for MKVExtract to extract the Lossless audio for conversion to FLAC. During this extraction process I've discovered some odd behaviour from both MediaInfo and MKVExtract, at one point MediaInfo failed to detect a DTS-MA stream as DTS-MA so the extraction process failed, looking at the file with the MediaInfo GUI showed the Track to be simple DTS and it stayed that way until a re-ran the file through MKVPropEdit, afterward MediaInfo showed the track as DTS-MA and it extracted perfectly. This isn't MediaInfo related but at another point and ran the extractor batch, the command line built fine but MKVExtract extracted nothing. I just ignored these problems until I came across the file I currently have on my HDD. After running the file through MKVPropEdit and MetaX MediaInfo failed to see ANY tracks in the file at all despite the fact that it played perfectly in both VLC and Windows Media Player and MKVMerge could process the file fine. At that point I contacted the developer of MKVToolnix to discuss what might be going wrong, he created THIS FAQ for my benefit:

https://trac.bunkus.org/wiki/FAQ%3AAllTracksVanishedAfterMkvpropeditOrHeaderEditor

And told me the MKVExtract issue was probably being caused by me using MediaInfo Stream Order instead of MKVToolnix TrackID. So I ran the file through MKVMerge with the -I command and then used the MKVToolnix TrackID to try to extract a track from the file which MKVExtract failed to accomplish. I gave him a description of what happened and within hours, without even looking at my file he realised what was wrong, identified the bug in MKVExtract, fix it and gave me a link to the new version which I downloaded and used to extract the audio track. MKVExtract is fixed but I'm pretty sure it's not going to be that simple with MediaInfo. I have a 27.5 GB file on my HDD that MediaInfo insists has no tracks inside, I realise the insanity of trying to get that file to you but I'll try if you're willing. Other than that, since the problem is caused by the way the Header Editors choose to format what they write I have no way of recreating the problem in a smaller file as I have no direct control over the process. I've looked at the file in a HEX editor and am trying to understand what's inside but some of the formatting isn't intuitive and I can't find a good explanation for it, I'm waiting for the Matroska web site to reply to an email I sent asking for information. I could try to continue but first I should ask what you really need from me or if you even want to deal with the problem at all.

Discussion

  • Sorry, the Milestone should be Incorrect Result and there's a bunch of mistakes in the text but I can't find an edit option.

     
  • And told me the MKVExtract issue was probably being caused by me using MediaInfo Stream Order instead of MKVToolnix TrackID.

    MediaInfo provides both Stream Order (request from MKVToolnix users) and TrackID, doesn't it? So you can choose.

    MKVExtract is fixed but I'm pretty sure it's not going to be that simple with MediaInfo.

    Currently, nothing is obvious to me (I am less expert in MKV than him :) ).

    I have a 27.5 GB file on my HDD that MediaInfo insists has no tracks inside, I realise the insanity of trying to get that file to you but I'll try if you're willing.

    It is not insane on my side, I already received several files a lot bigger than that ;-).
    If you can create a smaller file with the same issue, it is better, else you can provide it as is, I can provide a private FTP server if you need one (1 Gbps), please drop an email to info@mediaarea.net

    I could try to continue but first I should ask what you really need from me or if you even want to deal with the problem at all.

    I need all files you are talking about.

    Sorry, the Milestone should be Incorrect Result and there's a bunch of mistakes in the text but I can't find an edit option.

    Isn't there an edit button on the right of the title?

     
  • On the right of the title is 'Follow this ticket' and 'Unsubscribe from this ticket' but I don't see an edit.

     
  • OK, the file has finished uploading. It's been a while since the thing was created and I've learned a few things that make me doubt the problem is with MediaInfo, but I can't know. This file has been processed with both MetaX AND MKVPropEdit, recently I sent a smaller file to mosu from MKVToolNix that MetaX had created. Apparently one of the pointers MetaX had created was slightly off and it had written it's tags in the middle of a void and was reading them from there too, even though no other program could see them. Also MKVMerge is incapable of seeing any image MetaX has put in a file, VLCs ability to see these images is hit and miss yet MediaInfo seems to realise they're there without failure (of course it doesn't display them). The developer of MetaX has assured me he's put his processes through MKV test suites and they are fine. I don't really know what's going on. I didn't know at the time but this may devolve in to a simple exercise in compensating for someone else's errors, or not. However, MKVToolNix (MKVMerge) can see all the tracks in this file. VLC can play the file fine as can Windows Media Player with the latest Shark 007 codec installed. MKVExtract had problems with the file at first but selur found an actual bug in MKVExtract in the way it handled track info at the end of a file and after he corrected it it handled the file fine. MediaInfo can not find a single track in this file, as far as it's concerned other than a few tags and chapters this file is empty. If you tell me one of the pointers is wrong and that's the problem I'll believe it, if you tell me the track info is sitting in the middle of a void I won't argue. But you might possibly find something lacking in MediaInfo and it may be worth a look.

    I'll send this via email as well, just in case.

     
  • I'm wondering if there's an update on this. The file I gave was huge and I realise examining it would be a pain in the arse. MKVPropEdit is causing this problem when TAGs are first added to a file, EVERY TIME now, so I can pretty much recreate it in any file size I want. I've finally figured out how to use the MKVInfo GUI and can see what's causing it now. There's a single seek head pointer at the beginning of the files that points to series' of more specific seek pointers at the ends of the files and MediaInfo doesn't seem to recognise the configuration. I realise you have a lot to do and this isn't that important in the grand scheme of things, and my programming skills are rather rudimentary so I can't be of much help, but I'd like to know if the problem has at least been identified properly so I know it's just a matter of waiting for you to find the time. (Or would you actually like another file to show the issue?)

     
  • There's a single seek head pointer at the beginning of the files that points to series' of more specific seek pointers at the ends of the files and MediaInfo doesn't seem to recognise the configuration.

    Right.

    it's just a matter of waiting for you

    Right.
    Unfortunately, no other programmer is interested in patching this issue, and my sponsors have priority for the moment (and they yell too because I am also late with their requests :) )

     
  • Cool, I've had to put up with this every day for so long, I'll probably miss it when it's gone :P

     
  • I quickly checked the issue, I patched a bit my code.
    This version should be a bit more faster.

     
  • I'm not sure if you posted in the right place, 'faster' doesn't seem relevant to my issue, but I tried out the new version of MediaInfo and nothing has changed. These are the relevant headers as shown by MKVInfo:

    (MKVInfo) + EBML head at 0 size 40
    (MKVInfo) |+ EBML version: 1 at 5 size 4
    (MKVInfo) |+ EBML read version: 1 at 9 size 4
    (MKVInfo) |+ EBML maximum ID length: 4 at 13 size 4
    (MKVInfo) |+ EBML maximum size length: 8 at 17 size 4
    (MKVInfo) |+ Doc type: matroska at 21 size 11
    (MKVInfo) |+ Doc type version: 2 at 32 size 4
    (MKVInfo) |+ Doc type read version: 2 at 36 size 4
    (MKVInfo) + Segment, size 1036820732 at 40 size 1036820744
    (MKVInfo) |+ Seek head at 52 size 22
    (MKVInfo) | + Seek entry at 57 size 17
    (MKVInfo) | + Seek ID: 0x11 0x4d 0x9b 0x74 (KaxSeekHead) at 60 size 7
    (MKVInfo) | + Seek position: 1036820628 at 67 size 7
    ...
    (MKVInfo) |+ Tags at 1036820028 size 652
    (MKVInfo) | + Tag at 1036820034 size 92
    (MKVInfo) | + Targets at 1036820037 size 20
    (MKVInfo) | + TargetTypeValue: 70 at 1036820040 size 4
    (MKVInfo) | + TargetType: COLLECTION at 1036820044 size 13
    (MKVInfo) | + Simple at 1036820057 size 38
    (MKVInfo) | + Name: TITLE at 1036820060 size 8
    (MKVInfo) | + String: Beast Machines at 1036820068 size 17
    (MKVInfo) | + TagLanguage: eng at 1036820085 size 6
    (MKVInfo) | + DefaultLanguage: 1 at 1036820091 size 4
    (MKVInfo) | + Simple at 1036820095 size 31
    (MKVInfo) | + Name: TOTAL_PARTS at 1036820098 size 14
    (MKVInfo) | + String: 6 at 1036820112 size 4
    (MKVInfo) | + TagLanguage: eng at 1036820116 size 6
    (MKVInfo) | + DefaultLanguage: 1 at 1036820122 size 4
    (MKVInfo) | + Tag at 1036820126 size 83
    (MKVInfo) | + Targets at 1036820129 size 16
    (MKVInfo) | + TargetTypeValue: 60 at 1036820132 size 4
    (MKVInfo) | + TargetType: SEASON at 1036820136 size 9
    (MKVInfo) | + Simple at 1036820145 size 32
    (MKVInfo) | + Name: PART_NUMBER at 1036820148 size 14
    (MKVInfo) | + String: 01 at 1036820162 size 5
    (MKVInfo) | + TagLanguage: eng at 1036820167 size 6
    (MKVInfo) | + DefaultLanguage: 1 at 1036820173 size 4
    (MKVInfo) | + Simple at 1036820177 size 32
    (MKVInfo) | + Name: TOTAL_PARTS at 1036820180 size 14
    (MKVInfo) | + String: 22 at 1036820194 size 5
    (MKVInfo) | + TagLanguage: eng at 1036820199 size 6
    (MKVInfo) | + DefaultLanguage: 1 at 1036820205 size 4
    (MKVInfo) | + Tag at 1036820209 size 471
    (MKVInfo) | + Targets at 1036820213 size 17
    (MKVInfo) | + TargetTypeValue: 50 at 1036820216 size 4
    (MKVInfo) | + TargetType: EPISODE at 1036820220 size 10
    (MKVInfo) | + Simple at 1036820230 size 40
    (MKVInfo) | + Name: TITLE at 1036820233 size 8
    (MKVInfo) | + String: The Reformatting at 1036820241 size 19
    (MKVInfo) | + TagLanguage: eng at 1036820260 size 6
    (MKVInfo) | + DefaultLanguage: 1 at 1036820266 size 4
    (MKVInfo) | + Simple at 1036820270 size 32
    (MKVInfo) | + Name: PART_NUMBER at 1036820273 size 14
    (MKVInfo) | + String: 01 at 1036820287 size 5
    (MKVInfo) | + TagLanguage: eng at 1036820292 size 6
    (MKVInfo) | + DefaultLanguage: 1 at 1036820298 size 4
    (MKVInfo) | + Simple at 1036820302 size 340
    (MKVInfo) | + Name: DESCRIPTION at 1036820306 size 14
    (MKVInfo) | + String: Optimus finds himself back on a deserted Cybertron, being chased by an army of mindless tanks. Locked in his original beastmode by a virus, and suffering from amnesia, his only option is to flee. One by one he encounters his old friends, and it becomes apparent that something is leading them... somewhere... at 1036820320 size 312
    (MKVInfo) | + TagLanguage: eng at 1036820632 size 6
    (MKVInfo) | + DefaultLanguage: 1 at 1036820638 size 4
    (MKVInfo) | + Simple at 1036820642 size 38
    (MKVInfo) | + Name: CONTENT_TYPE at 1036820645 size 15
    (MKVInfo) | + String: TV Show at 1036820660 size 10
    (MKVInfo) | + TagLanguage: eng at 1036820670 size 6
    (MKVInfo) | + DefaultLanguage: 1 at 1036820676 size 4
    (MKVInfo) |+ Seek head at 1036820680 size 104
    (MKVInfo) | + Seek entry at 1036820685 size 17
    (MKVInfo) | + Seek ID: 0x1c 0x53 0xbb 0x6b (KaxCues) at 1036820688 size 7
    (MKVInfo) | + Seek position: 1036775473 at 1036820695 size 7
    (MKVInfo) | + EbmlVoid (size: 19) at 1036820702 size 21
    (MKVInfo) | + Seek entry at 1036820723 size 14
    (MKVInfo) | + Seek ID: 0x10 0x43 0xa7 0x70 (KaxChapters) at 1036820726 size 7
    (MKVInfo) | + Seek position: 87 at 1036820733 size 4
    (MKVInfo) | + Seek entry at 1036820737 size 15
    (MKVInfo) | + Seek ID: 0x15 0x49 0xa9 0x66 (KaxInfo) at 1036820740 size 7
    (MKVInfo) | + Seek position: 376 at 1036820747 size 5
    (MKVInfo) | + Seek entry at 1036820752 size 15
    (MKVInfo) | + Seek ID: 0x16 0x54 0xae 0x6b (KaxTracks) at 1036820755 size 7
    (MKVInfo) | + Seek position: 555 at 1036820762 size 5
    (MKVInfo) | + Seek entry at 1036820767 size 17
    (MKVInfo) | + Seek ID: 0x12 0x54 0xc3 0x67 (KaxTags) at 1036820770 size 7
    (MKVInfo) | + Seek position: 1036819976 at 1036820777 size 7

    This is what the file looks like in a file remuxed and reordered with MKVMerge:

    Unique ID : 223652146264883547830740127321852399791 (0xA841D628EA09B6B09A18509634AAD0AF)
    Complete name : F:\Videos\MKV\Beast Machines\Season 01\Beast Machines - 01x01 - The Reformatting.mkv
    Format : Matroska
    Format version : Version 4 / Version 2
    File size : 988 MiB
    Duration : 20mn 36s
    Overall bit rate mode : Variable
    Overall bit rate : 6 701 Kbps
    Movie name : The Reformatting
    Part/Position : 01
    Encoded date : UTC 2014-04-21 10:48:00
    Writing application : mkvmerge v6.9.1 ('Blue Panther') 64bit built on Apr 18 2014 18:23:38
    Writing library : libebml v1.3.0 + libmatroska v1.4.1
    TOTAL_PARTS : 22
    DESCRIPTION : Optimus finds himself back on a deserted Cybertron, being chased by an army of mindless tanks. Locked in his original beastmode by a virus, and suffering from amnesia, his only option is to flee. One by one he encounters his old friends, and it becomes apparent that something is leading them... somewhere...
    CONTENT_TYPE : TV Show

    This is what the original MKVPropEdit edited version looks like:

    Unique ID : 218117358920909628570088166812672169994 (0xA417E0133E869F3C133198168075980A)
    Complete name : E:\Make\Beast Machines\Season 01\Beast Machines - 01x01 - The Reformatting.mkv
    Format : Matroska
    Format version : Version 2
    File size : 989 MiB
    Duration : 20mn 36s
    Overall bit rate mode : Variable
    Overall bit rate : 6 708 Kbps
    Movie name : Beast Machines - 01x01 - The Reformatting
    Encoded date : UTC 2014-04-08 02:36:08
    Writing application : MakeMKV v1.8.8 win(x64-release)
    Writing library : libmakemkv v1.8.8 (1.3.0/1.4.1) win(x64-release)

    The Chapters and Track info is still shown in this file because they're at the beginning, but the Tags are completely missing. VLC sees the Tags fine.

     
  • I'm not sure if you posted in the right place, 'faster' doesn't seem relevant to my issue,

    Crap, sorry I mixed this ticket with another one :(.
    I did not yet work on tracks at the end (I see them. But I did not yet change the code for that)