|
From: Anton A. <ai...@ca...> - 2001-07-09 18:49:39
|
What do people think should be done if a damaged boot sector is found by the TNG NTFS driver but a valid backup boot sector is found? There are three possibilities, each bringing its own advantages and disadvantages: 1. As it is at the moment, the primary boot sector would be overwritten (unless mounted read only, etc) by the found backup copy. Advantage: we repair the fs silently. Disadvantage: if the user has just formatted their NTFS partition into an ext2 partition then chances are that the ntfs backup boot sector is still present and we would in that case corrupt the new ext2 super block overwriting it with the NTFS one. 2. Don't repair the primary boot sector but do use the backup one and continue attempt to mount. Advantage: a damaged NTFS partition would have a chance of working. Disadvantage: In the same example as above with the reformat of NTFS to ext2, chances are that either the MFT itself or the MFT mirror are still present so we could probably mount the partition ok before finding some invalid data so we could potentially again either damage data or just succeed with the mount even though we should have failed. This would for example cause the mounting of the VFS root to fail altogether because NTFS would succeed to mount but would then crash out with a corrupt fs error. 3. Find but do not use the backup boot sector. Just tell the user there is one and they should run chkdsk to recover it if this is what they want to happen. Advantage: We don't corrupt anything and the VFS mount will work. Disadvantage: A genuine, corrupt NTFS partition will not be mounted as the boot sector will not be found. We currently have 1. implemented but IMO it is too dangerous. We should probably settle for 3. to be real safe and maybe we should have a mount option to allow 2. to happen. Comments? Opinions? Anton -- "Nothing succeeds like success." - Alexandre Dumas -- Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @) Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/ ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/ |
|
From: Richard R. <nt...@fl...> - 2001-07-09 23:05:35
|
Hi Anton, On 09 Jul 2001 19:50:12 +0100, Anton Altaparmakov wrote: > What do people think should be done if a damaged boot sector is found by > the TNG NTFS driver but a valid backup boot sector is found? Just some comments... > 1. repair the fs silently Agreed, this is a bad idea. All repairs should be the jobs of chkdsk. > 2. use the backup one and attempt to mount. I'd like to think of it not as "might find some invalid data", but instead "might be able to recover at least something". This really is necessary to recover anything we can from a damaged fs. ext2 for example will allow you to use a backup superblock for recovery purposes. We'd just be doing the same thing. > mke2fs over NTFS? Been there, done that. It's interesting to note how LITTLE ext2 needs to represent an empty filesystem. Taking things further, how much do we actually NEED to recover data? $Boot (either of two copies) $MFT (need the first six entries (need root dir)) dot (any valid index allocations are a bonus) Things we can live without $LogFile $BadClus $UpCase (invalid -> bad collation (not serious)) $Volume $Bitmap (not actually necessary for a read-only driver) Then finally there's $AttrDef which we ought to read but don't actually take any notice of. So give me about 10 clusters of my choice I can give you a filesystem :-) You'd have to make the driver VERY resilient to read complete garbage, but thanks to the fixups we can figure out what's safe to read and what isn't. > 3. backup boot sector. run chkdsk to recover Yeah. Notify the user and don't touch anything. FlatCap (Rich) nt...@fl... |
|
From: Anton A. <ai...@ca...> - 2001-07-09 23:49:35
|
Hi Richard, At 00:05 10/07/2001, Richard Russon wrote: >On 09 Jul 2001 19:50:12 +0100, Anton Altaparmakov wrote: > > What do people think should be done if a damaged boot sector is found by > > the TNG NTFS driver but a valid backup boot sector is found? > >Just some comments... > > > 1. repair the fs silently > >Agreed, this is a bad idea. All repairs should be the jobs of chkdsk. Ok. Already removed in my local cvs tree. Will commit shortly. But I don't agree about "all repairs", see end of message. > > 2. use the backup one and attempt to mount. > >I'd like to think of it not as "might find some invalid data", but instead >"might be able to recover at least something". This really is necessary >to recover anything we can from a damaged fs. Yes, but the driver shouldn't worry about that kind of damage. A to be written ntfsck should do recovery not the driver, I think we agree on that from your comments on point 1, above. >ext2 for example will allow you to use a backup superblock for recovery >purposes. We'd just be doing the same thing. Yes. So you agree we shouldn't try to use any backups without an explicit mount option. That saves us from all dangers when mounting, then. > > mke2fs over NTFS? > >Been there, done that. It's interesting to note how LITTLE ext2 needs to >represent an empty filesystem. (-: If you think about it NTFS doesn't need that much space either. If you are talking huge volumes then the overhead is a really small fraction but I guess ext2 beats NTFS any day (but it's not a journalling fs so the comparison is unfair). >Taking things further, how much do we actually NEED to recover data? > > $Boot (either of two copies) > $MFT (need the first six entries (need root dir)) > dot (any valid index allocations are a bonus) You really need the whole MFT. Anything missing from the mft means trouble. But the list is shorter: dot is not needed at all. What for? It only contains copies of info already present in the mft records. Assuming you have a complete and not corrupt MFT, you can reconstruct a whole NTFS 1.2 volume including the directory tree. If the mft is corrupt then you would need the directory tree and the bitmap to be able to resolve what is corrupt and what not... $volume is quite important in that context to know which ntfs version you are dealing with so you know how to deal with some structures. >Things we can live without > > $LogFile > $BadClus > $UpCase (invalid -> bad collation (not serious)) > $Volume > $Bitmap (not actually necessary for a read-only driver) Agreed. >Then finally there's $AttrDef which we ought to read but don't actually >take any notice of. Windows doesn't read the file either as far as I can tell. It is completely pointless. You only need it to support ntfs volumes of different versions read-only. But Microsoft has taken the approach of not doing that, either. Ever tried reading a NTFS 3.x volume using NT 4.0 before SP4? It won't even boot. Compatibility. Ha! All attributes below 0x1000 are Microsoft owned (and starting with a $) so they are defined to be certain things and you can rely on that (well, they did change some attributes but considering their old counterparts were never used it doesn't really matter). It is irrelevant what attrdef says about them as this stuff is hard coded into the Windows driver. - The only advantages of the attrdef I can see is to check that we stay within the valid size limits for attributes but then again the only attributes where we need to care are the file name and the volume name so we can just as well hard code them with a few #defines somewhere in include/linux/ntfs_fs.h. All the others are either unbounded, fixed size or we don't ever touch them anyway. >So give me about 10 clusters of my choice I can give you a filesystem :-) Yes, that pretty much does it. >You'd have to make the driver VERY resilient to read complete garbage, >but thanks to the fixups we can figure out what's safe to read and what >isn't. Yes. > > 3. backup boot sector. run chkdsk to recover > >Yeah. Notify the user and don't touch anything. Yes. That's my preference, too for the backup boot sector. That way we have maximum safety. And if the user tells the driver to use the backup boot sector then we _know_ this is a ntfs fs and can start writing/fixing things. However, I do think the driver needs hotfix capabilities for many things. For example fixing inconsistencies between mft and mft mirror on mount. Or do you think the driver should just crash out in some way when it sees that the fs is corrupt? - I would much rather prefer the driver to try to recover the fs if possible (and mounted r/w). - The mft mirror mismatch is I guess something we could give up on and require chkdsk/ntfsck to be run but that would appear a very silly thing to do. And it gets much harder to bail out once we have a mounted and functional fs and corruption is encountered. If the corruption is minor we might very well be able to recover internally, no need for chkdsk, no need to crash the kernel, just a warning message that a hotfix was applied. Having said all that, that could be applied to the boot sector as well. So if the -o repair option is given on mount or something like that (-o damaged perhaps is better?), than the driver will do it's best to mount a damaged fs and recover as much data as possible. This can include fixing the boot sector. It's not exactly many lines of code for that particular code... If we say that kind of stuff is for chdkdsk to do then we can't really mount a damaged partition at all or we will run into trouble. As you say we would have to have massive amounts of checks to be able to cope with complete garbage data... The fixups unfortunately can be fine and the underlying data can be crap. It happens. Even the windows driver sometimes causes that kind of corruption to a ntfs volume... Anton -- "Nothing succeeds like success." - Alexandre Dumas -- Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @) Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/ ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/ |
|
From: Richard R. <ri...@fl...> - 2001-07-10 01:38:06
|
Hi Anton, > But I don't agree about "all repairs", see end of message. I suppose it's a fine line for some possible fixes. > > ext2 for example will allow you to use a backup superblock for recovery > > purposes. We'd just be doing the same thing. > Yes. So you agree we shouldn't try to use any backups without an explicit > mount option. That saves us from all dangers when mounting, then. Yeah, let the user decide how much damage we inflict. > > Taking things further, how much do we actually NEED to recover data? > You really need the whole MFT. > But the list is shorter: dot is not needed at all. What for? It only > contains copies of info already present in the mft records. If the volume gets corrupted, them for a file as large as the MFT, we're going to lose a few records. But you're right, that's the definitive reference. As long as we can find the runlists for the MFT we're OK. > Assuming you have a complete and not corrupt MFT, you can reconstruct > a whole NTFS 1.2 volume including the directory tree. Rebuilding the directory tree would NOT be pleasant. Userland/ntfsck? Fair enough. > If the mft is corrupt then you would > need the directory tree and the bitmap to be able to resolve what is > corrupt and what not... in that context to know > $Volume is quite important...which ntfs version Do we care about this? If we encounter an ungraded volume we could find either small (old) attributes or larger (new) ones. The code would have to check the attribute's size anyway. As for the new metadata, well we can't really do much with it yet :-) > > $AttrDef > It is completely pointless. :-) At least M$ don't still waste 36K on it. > > > 3. backup boot sector. run chkdsk to recover > I do think the driver needs hotfix capabilities for many things. OK, go on... > For example fixing inconsistencies between mft and mft mirror on mount. That's fairly crucial. > Or do you think the driver should just crash out in some way when it sees that > the fs is corrupt? Hey yeah, lets oops^U There may be another solution. How anti-social would it be to become read-only on a serious error. We find a problem that we can't solve without user intervention, so we log everything we know, and "remount" ourselves read-only. > I would prefer the driver to try to recover the fs if possible We'd have to start listing the possible problems. For example: If $Bitmap has a bit reset (0) when we KNOW it is in use it makes sense for us to correct it. If it has a bit set (1) when we think it shouldn't be we can't do anything about it. (chkdsk should find the orphan and save it to a file). We could end up with a filesystem with a config file :-) > If the corruption is minor we might very well be able to recover internally > warning message that a hotfix was applied. > Having said all that, that could be applied to the boot sector as well. So Yep, lots of logging. > -o repair / -o damaged ... the driver will do it's best to mount a damaged > fs and recover as much data as possible. repair sounds less negative, sounds like we know what we're doing. > The fixups unfortunately can be fine and the underlying data can be crap. > It happens. Even the windows driver sometimes causes that kind of corruption > to an ntfs volume... Argh! That would be harder to spot. At least it will be obvious when someone else (mke2fs) has trashed our volume. FlatCap (Rich) nt...@fl... |
