linux-ntfs-dev — Development discussions and bug reports.

Re: [Linux-NTFS-Dev] [PATCH] load_attribute_list overflow

[Szakacsits S. <sz...@si...>]

On Mon, 14 Apr 2003, Anton Altaparmakov wrote:

> > 2) I can't see the point checking for LCN_RL_NOT_MAPPED in
> >    do_partial/do_final. If earlier a same is found then it's
> >    error however in do_partial/do_final it's OK?
>
> Not sure what you mean. Look at what index to rl[] is being used. It is
> only a sanity check to make sure that either the run list is finished at
> the next element (rl[1])

Yes, this part is right.

> or if it is not finished then, it is allowed to be an
> unmapped/unallocated element (rl[1])

This is what I was confused. I thought decompress_mapping_pairs() can't
return LCN_RL_NOT_MAPPED but now I cheked and it can. So it's OK.

	Szaka

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Anton A. <ai...@ca...>]

On Mon, 14 Apr 2003, Szakacsits Szabolcs wrote:
> > An additional complexity is that your imaging program _has_ to cope both
> > with volumes where the backup bootsector is at the end of the partition
> > and with volumes where it is in the middle of the volume. - Yes these do
> > exist (with NT4-pre-SP4 and NT3).
> 
> I'm not sure I've seen this between NT4 and NT4-pre-SP4 but I believe NT3.
> I also think this case (backup bootsector in the middle of volume) could be
> just unsupported, NTFS version <= 1.1? I don't think it's worth bothering
> with this case since statistically nobody uses them.

NT3.51 has NTFS 1.2 _and_ it has the backup bootsector in the middle. The
same goes for NT4 before SP4 (before NTFS3.x support was added). I admit I
have never verified the NT4 before SP4 case but NT3.51 I have verified
personally (I have it installed in vmware).

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Szakacsits S. <sz...@si...>]

On Mon, 14 Apr 2003, Anton Altaparmakov wrote:
> This is completely not true. NTFS doesn't care how big the partition is
> (except for the backup boot sector which can be just left out to be
> replaced by chkdsk). The volume can be much smaller than the partition and
> it all works just fine.

True. As I've wrote already on Linux-NTFS-Dev the minimum partition
size needed is

        <number_of_clusters> * <cluster_size> + 512 bytes.

If [system] partition is smaller than this then Windows will just reboot
during boot without any message. Several people reported this to me who
shrunk the partition in the middle of NTFS after ntfsresize. When they
fixed (properly enlarged) the partition size eveything started to work
fine again.

This is also the reasons I can recommend people to do NTFS shrinking at
present as

	ntfsresize (Linux)
	defrag     (Windows)
	ntfsresize (Linux)
	fdisk      (Linux)
	chkdsk     (Windows)

because the built-in Windows defragmenter doesn't move file data towards
the beginning of the partition, it just defragments and actually sometimes
it even moves data towards the end of the partition.

> An additional complexity is that your imaging program _has_ to cope both
> with volumes where the backup bootsector is at the end of the partition
> and with volumes where it is in the middle of the volume. - Yes these do
> exist (with NT4-pre-SP4 and NT3).

I'm not sure I've seen this between NT4 and NT4-pre-SP4 but I believe NT3.
I also think this case (backup bootsector in the middle of volume) could be
just unsupported, NTFS version <= 1.1? I don't think it's worth bothering
with this case since statistically nobody uses them.

	Szaka

Re: [Linux-NTFS-Dev] [PATCH] load_attribute_list overflow

[Anton A. <ai...@ca...>]

On Sun, 13 Apr 2003, Szakacsits Szabolcs wrote:
> There is a bug in load_attribute_list [both 2.4 and 2.5 kernels]. If the
> initialized_size of an attribute list attribute is a multiply of block_size
> but not multiply of cluster size then overflow reported incorrectly and
> inode marked as bad however it's not true. The attached patch should fix it
> but I didn't test it. Thanks to Attila for helping to track down this
> problem [some files couldn't be read, I've seen same reports before also].

Thanks, applied/will apply.

> Two other comments on load_attribute_list():
> 
> 1) I don't agree with the sanity check at the function start is in #if
>    DEBUG. It should be there always.

Ok, removed debug check.

> 2) I can't see the point checking for LCN_RL_NOT_MAPPED in
>    do_partial/do_final. If earlier a same is found then it's
>    error however in do_partial/do_final it's OK?

Not sure what you mean. Look at what index to rl[] is being used. It is
only a sanity check to make sure that either the run list is finished at
the next element (rl[1]) or if it is not finished then, it is allowed to
be an unmapped/unallocated element (rl[1]) but in that case it must be
followed by the end with the element after that (rl[2]). In any other
cases from the ones above there is a corrupt attribute IMO, which is what
the check checks for. It also checks the code in so far that it makes sure
we always read to the end of the attribute. The check may be wrong, if you
think it is actually wrong rather than pointless please let me know.

Thanks for the fixes!

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Anton A. <ai...@ca...>]

On Sun, 13 Apr 2003, Ian Jackson wrote:
> But an NTFS volume cannot be moved to another partition using dd
> unless the destination partition is _exactly_ the same size as the
> source partition.

This is completely not true. NTFS doesn't care how big the partition is
(except for the backup boot sector which can be just left out to be
replaced by chkdsk). The volume can be much smaller than the partition and
it all works just fine.

An additional complexity is that your imaging program _has_ to cope both
with volumes where the backup bootsector is at the end of the partition
and with volumes where it is in the middle of the volume. - Yes these do
exist (with NT4-pre-SP4 and NT3).

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

[Linux-NTFS-Dev] [PATCH] load_attribute_list overflow

[Szakacsits S. <sz...@si...>]

Hi,

There is a bug in load_attribute_list [both 2.4 and 2.5 kernels]. If the
initialized_size of an attribute list attribute is a multiply of block_size
but not multiply of cluster size then overflow reported incorrectly and
inode marked as bad however it's not true. The attached patch should fix it
but I didn't test it. Thanks to Attila for helping to track down this
problem [some files couldn't be read, I've seen same reports before also].

Two other comments on load_attribute_list():

1) I don't agree with the sanity check at the function start is in #if
   DEBUG. It should be there always.

2) I can't see the point checking for LCN_RL_NOT_MAPPED in
   do_partial/do_final. If earlier a same is found then it's
   error however in do_partial/do_final it's OK?

	Szaka

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Ian J. <ia...@da...>]

Szakacsits Szabolcs writes ("Re: [Linux-NTFS-Dev] Implementing ntfsimage"):
> On Thu, 10 Apr 2003, Ian Jackson wrote:
> > So, perhaps I should have ntfsimage generate an image not containing
> > the spare boot sector, but with the `dirty' bit set, so that NT will
> > rewrite the spare ?
>
> I don't think so. For imaging that seems to be too intrusive.
>
> If you can access that sector just do it.

This is all very well, but the results can't be expressed in a
filesystem-neutral way.

Consider: the stream image format I'm planning will allow you to
efficiently represent, essentially, a sparse file.  This is good
enough for all `sensible' filesystems - ie, filesystems which can be
moved to another partition using `dd' provided that the destination
partition is big enough.

So the whole thing is some kind of optimised dd.

But an NTFS volume cannot be moved to another partition using dd
unless the destination partition is _exactly_ the same size as the
source partition.

I don't want to put filesystem-specific knowledge into the image
format - before you know it there'll be weird hacks to fix up CHS
information in FAT boot sectors.  I want the image format to be
exactly a representation of an optimised dd.

So that leaves precious few choices for an NTFS image generator.  I
can include the spare boot sector in the image stream, but that makes
it dependent on the exact partition size (which is at least something
that I think one might reasonably put a flag in the image stream
header for).  Or I can mark the filesystem dirty and not include the
spare boot sector (but still make the image say it needs 512 bytes
more than the core of the fs).

I think the best option is to take one of these two approaches at the
user's option.  By default probably mark the fs dirty, as that makes
the resulting image less fragile, with options to ignore the issue and
not mark the fs dirty (ie user promises to run chkdsk themselves), and
to make the image specific to the exact partition size.

(In the case of the exact partition size specification, you can of
course tell restoreimage to restore it anyway, but that's all getting
a bit hairy.)

> If not, you can either
>   a) warn the user, to run chkdsk to recovery it because Linux
>      kernel couldn't access that sector due to its limitation
> or
>   b) access the sector via the disk, not the partition (could be
>      quite messy/error-prone considering disk geometry issues)

I think the right answer if we want to access the sector and can't is
to fail and explain why.  If the user is troubled they can use the
other option - don't copy spare boot sector, mark fs as dirty.

> Anyway, don't waste time with this in the beginning, it's enough if you're
> aware of this issue when designing the format.

I think this issue is very difficult, as you can see.

Hrm, I think I may be a bit unclear here, so let me explain my current
proposal again slightly differently:

ntfsimage will have three modes:

 Mode                 --dirty        --promise-chkdsk    --exact-size
                      (default)      (unsafe)

  Dirty bit in         Always         Copied from real    Copied from real
  fs image in           dirty          filesystem volume   filesystem volume
  output                (changed)      (ie unchanged)      (ie unchanged)

  Image size as        fs size        fs size             Size of actual
  declared in image     + 512          + 512               partition
  stream header         bytes          bytes

  Image `exact size'   Clear (ie,     Clear (ie           Set (ie, exact
  flag in stream hdr    flexible)      flexible)           size required)

  Spare boot sector    Not present    Not present         Last 512 bytes
  location in image                                        of image

  If last sector       Irrelevant     Irrelevant          Fail and suggest
  can't be read?                                           use of --dirty

When you're restoring, this plays out like this:

  To restore to        No problem     No problem          No problem
  same size part'n     chkdsk runs    User runs chkdsk    No chkdsk needed

  To restore to        No problem     No problem          User must give
  different size       chkdsk runs    User must           --force option
                       automatically  run chkdsk          to restoreimage
                                      by hand             and run chkdsk

Ian.

Re: [Linux-NTFS-Dev] NTFS Support for RedHat 9 (Shrike)

[Richard R. <ri...@fl...>]

Hi Szaka,

Carelessly, I managed to lose your reply (after reading it), but it went
something like:

> Why, oh why do you hate RedHat so :-)

It's nothing specific, I just can't understand what they're up to.
ntfs has been a rough ride, lots of people have lost lots of data,
but it's fairly stable now, even the old driver.  The redhat engineers
agree.

Perhaps they really are worried about being sued.  Perhaps they think
that they're a big enough target for suing.  But of all the questionable
things in a linux distro they omit ntfs and mp3.  Surely they're trying
to get people to move over from windows?  OK, so you've installed redhat
for the first time and you want to look at your windows partiton.  You 
can't.  So you read the release notes.  rh8 had a clause that said, "oh,
btw, you can't see ntfs".  rh9 doesn't even have that.  You search
redhat's site for clues.  None.  wtf?  (actually, now they recommend
recompiling).  mp3s get slightly better treatment.

Perhaps I should hassle redhat more.  The last time I tried I was
ignored.  If they just acknowledged the existance of ntfs.  In a way it
doesn't matter too much.  Type ntfs into google and we come up first.
Lots of newbie rh users find us and find happiness.

/me sighs

Rich.

Re: [Linux-NTFS-Dev] ntfsclone added to bk and snapshots of ntfsprogs-devel

[Szakacsits S. <sz...@si...>]

On Fri, 11 Apr 2003, Anton Altaparmakov wrote:

> As above. Note that ntfsclone does not get built or distributed by default
> as it doesn't have a man page...

I'd agree even if it had a man page. It's in a very flux state now:
usage, invocation, internals, functionality (what metadata to dump and/or
wipe) and some important FIXME's.

But maybe it's worth to note how to compile. From the top dir of ntfsprogs:

	./configure
	make install
	cd ntfsprogs
	make ntfsclone
        [and the binary is in ".libs" directory]

The tar.bz2 snapshots from BitKeeper are at

	http://linux-ntfs.sourceforge.net/snapshots/

As usual comments, ideas are welcomed :)

	Szaka

[Linux-NTFS-Dev] ntfsclone added to bk and snapshots of ntfsprogs-devel

[Anton A. <ai...@ca...>]

As above. Note that ntfsclone does not get built or distributed by default
as it doesn't have a man page...

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Szakacsits S. <sz...@si...>]

On Thu, 10 Apr 2003, Ian Jackson wrote:
>
> This can be done, of course, just by piping the output of the imager
> into the restorer.  The few resulting extra memory copies will
> probably pale into insigificance next to the overhead of disk reads
> and writes.  Although, I'll try it and see.

I think you're right. If I guess correctly how you implement them, yours
will be even faster. The reason, ntfsclone is suboptimal for disk IO, it
will seek a lot but not yours. It's minutes to fix this and ntfsclone
should be faster. We will see then by how much ...

> > I think this is the wrong level. I really don't want to have to create an
> > image to be able to create a sparse file from that, that would take twice
> > the space on disk and twice the time.
>
> You don't have to store the image stream anywhere (indeed, you'd
> probably prefer not to for the reasons you say).

True but there is something I didn't mention before, see below ...

> Is the additional complexity in the imager worth it to avoid the need
> to run two commands (or have a shell alias) ?

It's not worth. This is why we need two tools. One for developers
(ntfsclone) and one for users (ntfsimage). But still see below :)

> Copying a sparse file to a partition isn't a restore, it's an
> image-and-restore.  So in my model you use the imager and restorer,
> just as before.

Doing the 'ntfsclone --metadata' (it's very important for NTFS development)
in this model is:

	image --metadata (and several more options)
	restore
	wiping not needed and resident user data (and several more options)

3 tools and confusions. I don't think one want to populate ntfsimage with
the --metadata stuff.

On the other hand, ntfsclone _is_ very inefficient for the purpose
ntfsimage is written.

Use the right tool for the right job! :)

	Szaka

[Linux-NTFS-Dev] metadata-only images

[Szakacsits S. <sz...@si...>]

On Thu, 10 Apr 2003, Anton Altaparmakov wrote:

> have the imaging program do it immediately on image creation. It would
> also be really nice if on restore the imaging program can take the sparse
> file and use that to restore the data to the partition. That allows for
> example to run ntfsck (that will one day be written) and then to just
> restore the metadata to the old state quickly by restoring the sparse
> image with the metadata only option enabled.

Nice idea. It feels _very_ dangerous but right now I don't have any
argument against it.

However doing so now, you would destroy _all_ resident user data and
timestamps. 'ntfsclone --metadata' wipes the first one due to protecting
users' privacy and the second one for better compressed metadata-only
images to be potentially transfered through WAN.

Of course we can have options to disable these for the price to create
bigger compressed metadata-only images. Actually I planned most or all
"wipe" things to be optional, just like many metadata ones as well. And
probably the --metadata default must be with all disabled.

In short, don't do the below at least for now because it will destroy your
data (due to misusing the tool it was designed for)

	ntfsclone --metadata -o file device
        doing things on file ...
	ntfsclone --metadata -o device file

Luckily the last line doesn't work and when I fix it probably I'll disallow
that combination.

	Szaka

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Szakacsits S. <sz...@si...>]

On Thu, 10 Apr 2003, Ian Jackson wrote:

> So, perhaps I should have ntfsimage generate an image not containing
> the spare boot sector, but with the `dirty' bit set, so that NT will
> rewrite the spare ?

I don't think so. For imaging that seems to be too intrusive.

If you can access that sector just do it. If not, you can either

  a) warn the user, to run chkdsk to recovery it because Linux
     kernel couldn't access that sector due to its limitation
or

  b) access the sector via the disk, not the partition (could be
     quite messy/error-prone considering disk geometry issues)

Anton has/had a kernel patch to fix this years ago but wasn't integrated
into the kernel for whatever reason.

> (I also need to set the image size to one cluster larger, to make sure
> that there will be space for the spare.)

Yes but as I've always seen, it's not one extra cluster. Only 512 bytes. So
the minimum size needed to restore is

	<number_of_clusters> * <cluster_size> + 512 bytes.

Anyway, don't waste time with this in the beginning, it's enough if you're
aware of this issue when designing the format.

	Szaka

Re: [Linux-NTFS-Dev] an option does not work for me.

[Szakacsits S. <sz...@si...>]

Hi Martin,

On Thu, 10 Apr 2003, Martin Dimitrov wrote:

> My name is Martin. I was trying to mount my NTFS partition automaticaly
> but there is something that did  not work. When I add this line to
> the /etc/fstab file:
> dev/hda1   /mnt/windows  ntfs ro  0 0
> it works, but if I try to add a user, for example (slice4e) :
> /dev/hda1   /mnt/windows  ntfs ro, uid=slice4e 0 0
> it does not work.

Don't leave space between "ro, uid=slice4e".

> Thank you very much for your time and help.

You're welcome :)

	Szaka

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Szakacsits S. <sz...@si...>]

On Thu, 10 Apr 2003, Szakacsits Szabolcs wrote:
> > I agree with Szakacsits: ntfsclone is a good name.
>
> ntfsclone - ntfsdump
>     2     :    1

      3     :    1

with Leonard's vote. I'll rename it and send a new patch to Anton, OK?

	Szaka

[Linux-NTFS-Dev] an option does not work for me.

[Martin D. <sl...@ho...>]

Hi ,

My name is Martin. I was trying to mount my NTFS partition automaticaly
but there is something that did  not work. When I add this line to
the /etc/fstab file:
dev/hda1   /mnt/windows  ntfs ro  0 0
it works, but if I try to add a user, for example (slice4e) :
/dev/hda1   /mnt/windows  ntfs ro, uid=slice4e 0 0
it does not work.

My NTFS driver is v1.1.22
Linux Version is 2.4.18-27.8.0
and NTFS - XP

I mounted ntfs manually. Then I tried to check my NTFS driver:
grep -i ntfs /var/log/messages
    I got this message :  Unknown option 'slice4e'
However I dont know if that is important since everything works when I do it 
manually.
My main question is if you can help me set it up so that I can
mount automaticaly adding a user.

Thank you very much for your time and help.

Martin Dimitrov




_________________________________________________________________
Protect your PC - get McAfee.com VirusScan Online  
http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Szakacsits S. <sz...@si...>]

On Thu, 10 Apr 2003, Ian Jackson wrote:

> I agree with Szakacsits: ntfsclone is a good name.

ntfsclone - ntfsdump
    2     :    1

More vote? Of course we can rename it ...

	Szaka

Re: [Linux-NTFS-Dev] [PATCH] ntfsdump

[Szakacsits S. <sz...@si...>]

WARNING! I've never needed the 'ntfsdump -[oO] device file' case and I'm
afraid it's broken, don't use it now!!!

Note2: after an ntfsdump you can check the dump with 'ntfsresize -i file'.

	Szaka

On Thu, 10 Apr 2003, Szakacsits Szabolcs wrote:

> Here it is. It's fully functional and works for me. But it's for 1.8 devel
> tree (should patch cleanly 1.7.1 source as well) and ALPHA code. There are
> lots of FIXME's in the source and also the internals, especially walk
> functions, will change dramatically. Now the control is an unmaintainable
> mess, there are kludges and a lot of duplicated code from ntfsresize.
>
> Just patch, autogen.sh and make install.
>
> Usage: ntfsdump [options] device
>
>     -o file --output file  Dump NTFS to the non-existent file
>     -O file                Dump NTFS to file, overwriting if exists
>     -m      --metadata     Dump *only* metadata (for NTFS experts)
>
> If -o is '-' then it dumps to stdout otherwise dumps to a sparse file. The
> sparse file should be packed with 'tar -Sjcf', that's what I found to be
> the most efficient from speed and compression point of view [if I remember
> correctly].
>
> With option --metadata user data is not dumped, only NTFS internal data is
> saved. Moreover for best compression, several things are also nullified (I
> plan more but almost all will be optional). As an example, here is a stat
> from a --metadata dumped volume. 4 GB originally and the compressed dump
> is only 1.5 MB, still mountable after uncompression).
>
> 	Num of MFT records       =    53353
> 	Num of used MFT records  =    53151
> 	Wiped unused MFT data    =   549275
> 	Wiped deleted MFT data   =    32971
> 	Wiped resident user data =  1688107
> 	Wiped timestamp data     =  3990400
> 	Wiped totally            =  6260753
> 	Compressed size          =  1571289
>
> In both cases [full or metadata-only dumps] the dumps are mountable, e.g.
>
> 	ntfsdump -o ntfs.dump device
> 	mount -o loop ntfs.dump /mnt/ntfs
>
> I've found the compressed metadata-only dumps are between 0.3 - 4 MB
> usually independently on volume size. The reason, it mostly depends on
> the number of MFT records in use. Most people have 30,000 - 200,000 files.
>
> Question, comments, ideas are welcomed. Note, I consider this mainly as a
> development tool, not for NTFS imaging Ian is working on.
>
> Cheers,
>
> 	Szaka
>

-- 

[Linux-NTFS-Dev] [PATCH] ntfsdump

[Szakacsits S. <sz...@si...>]

Hi,

Here it is. It's fully functional and works for me. But it's for 1.8 devel
tree (should patch cleanly 1.7.1 source as well) and ALPHA code. There are
lots of FIXME's in the source and also the internals, especially walk
functions, will change dramatically. Now the control is an unmaintainable
mess, there are kludges and a lot of duplicated code from ntfsresize.

Just patch, autogen.sh and make install.

Usage: ntfsdump [options] device

    -o file --output file  Dump NTFS to the non-existent file
    -O file                Dump NTFS to file, overwriting if exists
    -m      --metadata     Dump *only* metadata (for NTFS experts)

If -o is '-' then it dumps to stdout otherwise dumps to a sparse file. The
sparse file should be packed with 'tar -Sjcf', that's what I found to be
the most efficient from speed and compression point of view [if I remember
correctly].

With option --metadata user data is not dumped, only NTFS internal data is
saved. Moreover for best compression, several things are also nullified (I
plan more but almost all will be optional). As an example, here is a stat
from a --metadata dumped volume. 4 GB originally and the compressed dump
is only 1.5 MB, still mountable after uncompression).

	Num of MFT records       =    53353
	Num of used MFT records  =    53151
	Wiped unused MFT data    =   549275
	Wiped deleted MFT data   =    32971
	Wiped resident user data =  1688107
	Wiped timestamp data     =  3990400
	Wiped totally            =  6260753
	Compressed size          =  1571289

In both cases [full or metadata-only dumps] the dumps are mountable, e.g.

	ntfsdump -o ntfs.dump device
	mount -o loop ntfs.dump /mnt/ntfs

I've found the compressed metadata-only dumps are between 0.3 - 4 MB
usually independently on volume size. The reason, it mostly depends on
the number of MFT records in use. Most people have 30,000 - 200,000 files.

Question, comments, ideas are welcomed. Note, I consider this mainly as a
development tool, not for NTFS imaging Ian is working on.

Cheers,

	Szaka

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Ian J. <ia...@da...>]

Anton Altaparmakov writes ("Re: [Linux-NTFS-Dev] Implementing ntfsimage"):
> On Thu, 10 Apr 2003, Szakacsits Szabolcs wrote:
> > I think I've found the right name [first time in my life?]. ntfsclone.
> > Because actually that's what my tool does, clones. And hopefully this will
> > be also scray, unusual enough for people with real imaging need, not to
> > bother [mis]using it. The only problem, the metadata-only dumps fits the
> > name ntfsdump better ... Opinions? The name is the only showstopper to send
> > it for playing.
> 
> My vote would be ntfsdump.

I think a program called `ntfsdump' should do roughly the same thing
as `dump', which I don't think this program is supposed to (ie, a
file-level copy).

I agree with Szakacsits: ntfsclone is a good name.

Ian.

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Ian J. <ia...@da...>]

Szakacsits Szabolcs writes ("Re: [Linux-NTFS-Dev] Implementing ntfsimage"):
> If the boot sector gets damaged [quite often can happen, see many Microsoft
> KB's] you have a harder time to boot Windows. I can imagine Microsoft's
> Restore/Recovery or whatever it's called solution relies on it when
> people can not boot. It's not there without a reason.

Hrm.

> BTW, do you know that sometimes it even can not be done due to a Linux
> kernel bug? IMHO it's fixed in 2.5 but not earlier. For more see 'man
> mkntfs' BUGS section.

Even worse.

So, perhaps I should have ntfsimage generate an image not containing
the spare boot sector, but with the `dirty' bit set, so that NT will
rewrite the spare ?  (I also need to set the image size to one cluster
larger, to make sure that there will be space for the spare.)

Ian.

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Anton A. <ai...@ca...>]

On Thu, 10 Apr 2003, Szakacsits Szabolcs wrote:
> I think I've found the right name [first time in my life?]. ntfsclone.
> Because actually that's what my tool does, clones. And hopefully this will
> be also scray, unusual enough for people with real imaging need, not to
> bother [mis]using it. The only problem, the metadata-only dumps fits the
> name ntfsdump better ... Opinions? The name is the only showstopper to send
> it for playing.

My vote would be ntfsdump.

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Szakacsits S. <sz...@si...>]

> I'd also like but I'm afraid we must distinguish them. I renamed my
> ntfsimage to ntfsdump, so it will be easier to refer to them. At least
> temporarily and we can see later on if merge make sense or not.

I think I've found the right name [first time in my life?]. ntfsclone.
Because actually that's what my tool does, clones. And hopefully this will
be also scray, unusual enough for people with real imaging need, not to
bother [mis]using it. The only problem, the metadata-only dumps fits the
name ntfsdump better ... Opinions? The name is the only showstopper to send
it for playing.

	Szaka

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Szakacsits S. <sz...@si...>]

On Thu, 10 Apr 2003, Ian Jackson wrote:
> Szakacsits Szabolcs writes ("Re: [Linux-NTFS-Dev] Implementing ntfsimage"):
> > Yes. In your case [and quite probably in many others]. But the partition
> > can be much bigger than the NTFS volume size and chkdsk always creates the
> > backup boot sector in the last 512 bytes of the partition. Believe me, it
> > has nothing to do with cluster bitmap. Your case is pure accident.
>
> Oh, bloody hell.  What happens if it's missing ?

If the boot sector gets damaged [quite often can happen, see many Microsoft
KB's] you have a harder time to boot Windows. I can imagine Microsoft's
Restore/Recovery or whatever it's called solution relies on it when
people can not boot. It's not there without a reason.

One of the main reasons ntfsresize marks the volume dirty is to schedule
'chkdsk' immediately at boot to restore the backup boot sector. ntfsresize
can not know where users and partitioning tools will resize the partition.
I wrote about the issue here, if interested

	http://www.geocrawler.com/archives/3/8806/2002/11/0/10263407/

With parted this won't be a problem. Also for volume enlargement. But IMHO
you should definitely consider restoring it.

BTW, do you know that sometimes it even can not be done due to a Linux
kernel bug? IMHO it's fixed in 2.5 but not earlier. For more see 'man
mkntfs' BUGS section.

	Szaka

Re: [Linux-NTFS-Dev] Implementing ntfsimage

[Szakacsits S. <sz...@si...>]

On Thu, 10 Apr 2003, Anton Altaparmakov wrote:

> I agree that a simple image format is a good idea but I also think that
> being able to create a simple sparse file for debugging purposes
> especially a metadata only file is extremely valuable, too. If we can

Yes, the problem developers' and imaging users' need/requirements differ.

> combine the two utilities that would probably be the best thing. My
> suggestion would be to create one utility rather than two different ones

I'd also like but I'm afraid we must distinguish them. I renamed my
ntfsimage to ntfsdump, so it will be easier to refer to them. At least
temporarily and we can see later on if merge make sense or not.

ntfsdump really looks like a developer tool. I plan a lot of options to
control what metadata to be save or dropped. These could just confuse
users. These are needed to protect users' private info (I wiped all user
data except filenames but I plan later on that one as well, maybe) and
balance between saved metadata information and the dumped/compressed sized
for fast/easy transfer over network even if the volume is 200 GB. More
later.

Also confusing how to restore. With ntfsdump it's only ntfsdump. With
ntfsimage it's ntfsimage/restoreimage. And if combined? Uhmm.

I think ntfsdump would be useful for developers. Telling the user,
e.g. "please do this and send the foo.tar.bz2 file"

	ntfsdump -maQpLD -o foo device; tar -Scjf foo.tar.bz2 foo

It's just an extra bonus it can be also used as a backup tool [hell, I
definitely don't want to write/maintain a full featured imaging tool ;)].

And users with real imaging need can use ntfsimage and restoreimage. Easy
usage. Make the image with ntfsimage, restore it with restoreimage
(ntfsrestore could be a hard link ...)

> Yes, of course. But ideally I would like to see the two functionalities in
> one rather than having two separate imaging programs, one for sparse
> files and one for custom image format...

I think we shouldn't confuse users with this sparse thing. If we do sparse
it should be transparent. But we will see ...

> >  - restoreimage should be able to create the image as a sparse file. I
> >    don't think this would be a big issue.

> I think this is the wrong level. I really don't want to have to create an
> image to be able to create a sparse file from that, that would take twice
> the space on disk and twice the time. Not worth it IMHO.

True. Space is pretty important for me since I don't have much. The time is
also :) I don't have much from that either :)

> Much better to have the imaging program do it immediately on image
> creation. It would also be really nice if on restore the imaging
> program can take the sparse file and use that to restore the data to
> the partition. That allows for example to run ntfsck (that will one day
> be written) and then to just restore the metadata to the old state
> quickly by restoring the sparse image with the metadata only option
> enabled.

I can also do _now_

	ntfsdump -o foo device_big
	ntfsresize foo
	ntfsdump -o device_small foo

With ntfsimages/restoreimage it can't be done.

Both methods has advantages and disadvantages.

	Szaka