Non-1KB mft file records
LINUX based utility for use with gnuddrescue to aid with data recovery
Brought to you by:
maximus57
Menu
▾
▴
Hello again :)
today I got a disk from USB box that converts sector size from 512 bytes to 4KB; it has one ntfs partition (750GB) and value of sector size in boot sector was 4096, sectors per cluster was 1 and MFT record size was 4096 too! ddru_ntfsfindbad could not properly handle it (I think that you have hardcoded most common values there) so I report that and trying to attach bootsector and mft beginning here but see no option to do it...
Thanks in advance!
Here are attached files
Unfortunately I have not had a 4KB sector disk to test with. But I was assuming that the MFT records would still be 1K in size according to the standard, so ddru_ntfsfindbad was hardcoded that way. This will not be an easy fix at all. I don't even have a proper way to test it, and it is something that I cannot simulate. And as far as I can tell from your attachments, the full 4K is used in at least one the MFT records, so I can't accurately just short process the records. Thanks for reporting this, and I really like the included attachments. But I can give no timeline on when I may be able to address this properly. This is going on the hope-to-want-to-do list.
There is a possibility that maybe I could try to jump to the start of each record and let it short process as best possible. But this would be a hack, and I still would not be able to test it myself.
Anyway, you do a great work and cases like that are really rare :) but if in future you decide fo deal with it you'll not need to have disk with 4KB sector as all parameters (except of that is written in MBR) are simply "logical" like clusters in filesystem. All that including MFT file record size is taken from boot sector of partition. Sector size (2 bytes) is stored at offset 0x0b, sectors per cluster (1 byte) -- at offset 0x0d. MFT record size and index buffer size (1 byte each) are at 0x40 and 0x44 and are tricky encoded:
If this number is positive (up to 7F), then it represents clusters per record. If the number is negative (80 to FF), then the size of the file record is 2 raised to the absolute value of this number.
So, in my case at offset 0x40 is stored 0x1 that means "1 cluster per record", and in most cases there is 0xF6 that gives us -10 and so we have records 1024 (2¹⁰) bytes long.
As I still have that weird usb box I'll try to make ntfs partition on working disk and attach here full MFT file. Hope this can help in future :)
ps: maybe it is worth to just check mft record size in boot sector and exit with error message if it is not 1K?
Last edit: olegkrutov 2014-04-12
My software already gets sector size and sectors per cluster from the boot sector to calculate the cluster size, which is why there is no manual option for either. But I was not prepared for the change in MFT record size at all. There are many things hard coded to this.
I did not know this. Thank you for that information. I knew that setting was in the boot sector, but everything I had read up to this point was that MFT records were always 1024 bytes. I was not aware of information about how to process the actual value.
As for the memory usage, I had actually never looked at it until now. It is much higher than it should be. I don't have the most efficient method of memory usage, but I will now start thinking about how I can improve it in future releases. It needs plenty of memory because I allow for enough space for long filename lengths, and it loads all the MFT filenames into memory. This is done because I do not use the index records. Instead I rebuild the full path name from the inodes themselves, which requires having the whole list in memory (or it would take much much longer). This eliminates possible issues from corrupt index records. The sad thing is that the memory used seems to be about twice what loading a whole 1024 byte-per-record raw MFT would require.
I have just released 2.5beta1. It now should properly handle any sector or record size properly (up to 65536 bytes). I say should as I am limited to the testing I can do. I still don't have an actual drive with 4096 byte sectors. I used the magic of QEMU and G-Parted to create a (very) basic partition image file with 4096 byte sectors to test with. I had to figure out how to handle it on my own, as there is little to no information about how to process it.
I would not mind at all if you could put this to the test to see if it works properly on a real drive.
Last edit: maximus57 2014-05-25
That's a good news! I'll try it out at work. I have been looked into code and seen (if I haven't missed something) that you check fixup offset on value 0x30. But it can be not only 0x30 (filesystems created by winXP and later), but 0x2A too (created by NT, Win2K) (it looks like "FILE0" or "FILE*" signatures, respectively).
Unfortunately this is still hard coded in my software. The only thing different is that I realized the issue and included a safety catch that was not there before.
My problem is that 2C-2F is supposed to be the record number. If the fixup is at 2A then the record number can't be where it is, and I have no information on that. And I use the record number for verification.
Yes, the "old" NTFS with fixups at 0x2A does not have a record number at all. AFAIK the fixups are made for record validity verification so one can use it "officially" :) Moreover, record number is rather "undocumented" as it is not needed for functionality and can (but I have not seen that) be just not written by some third-party driver...
Version 2.5beta2 released. It will process if the fixup offset is 0x2A, although it only allows for the basic 512 byte sector / 1024 byte record size in that case. I can't see win2k or NT being able to handle anything other than that anyway.
Also I have downgraded the warning / error if the record number doesn't match, so it will still process now.