linux-ntfs-dev — Development discussions and bug reports.

Re: [Linux-NTFS-Dev] New libntfs API

[Szakacsits S. <sz...@si...>]

On Wed, 11 Dec 2002, Andrew Clausen wrote:

> * it should be possible for users of libntfs to pass their own IO
> abstraction (rather than using the kernel's or file system's)

Just curious, why do you need it? For me it would be very useful for
testing/debugging purposes.

> * error handling doesn't look to good :/

Yes, more info where errors happen (both DEBUG and release version),
debug levels (also both DEBUG and release version) and not mixing
stderr/stdout in DEBUG would be really nice(/important).

> * progress meters?

resizer has and I plan to keep it. However next resizer iteration
(cluster relocation support) uses two passes. First for cluster
accounting, sanity checks, building some data structures and the
second for doing the resize. No way to have one "smooth" progress
meter, the rate of the time spent in each pass can be any - of course
it could be done with only one if one drastical speed change
is acceptable.

> * all the functionality of userland tools should be in libntfs, or
> perhaps libntfs-mkfs, etc.  (This is already on your TODO :)

I'm already doing this for the resizer along with the cluster
relocation support, as time allows. But I don't plan to do for
other things.

> I think reiser4's (as opposed to reiser3's!) userland implementation
> is fantastic... take a look!
> 	www.namesys.com/snapshots/2002.12.04

Thanks, I will. I'm interested getting an ntfs resizer into parted
[it's one of the items in ntfsresize faq] .

	Szaka

Re: [Linux-NTFS-Dev] New libntfs API

[Richard R. <nt...@fl...>]

Hi Andrew,

> It looks like a good start :)  OTOH, I'd like to see lots of
> improvements:

Oh-oh :-)

> * it should be possible for users of libntfs to pass their own IO
> abstraction (rather than using the kernel's or file system's)

Pardon?  What do you want to replace?  The actual writing layer
at the bottom?  Interesting.

> * error handling doesn't look to good :/
> * progress meters?

Indeed.  Here are two things that could be improved without having
to know too much about NTFS.

> in libparted, I implemented progress meters by having
> all "big" functions accepting an extra parameter, "PedTimer* timer".

Nice idea, but I think it would have to be a low priority one for now.

> I think reiser4's userland implementation is fantastic... take a look!

I shall.

> Is it possible to make libntfs look like that?

Given time, yes.  Unfortunately, at the moment there's nobody with both
the skills and the free time.  Also, due to infighting, morale is at an
all time low.

> Is there any need to "stabilize" an API...

<paranoia>
Hmm... not unless Anton has some ulterior motive :-)
</paranoia>

> I think I'll have time to help with this stuff :)  (Uni vacation :)

Now we're talking.  That would be much appreciated.  If you have any
questions, don't hesitate to ask.  Over the xmas period, I'll check my
mail whenever I sober up.

Cheers,
  FlatCap (Rich)
  nt...@fl...

Re: [Linux-NTFS-Dev] hidden files?

[Richard R. <nt...@fl...>]

Hi farmerdude,

> http://patriot.net/~carvdawg/scripts/ads.pl

Ah, now that's what I wanted to know!  How to list the data streams
of files (without first knowing their names).

I shall have a play with BackupRead, BackupSeek and WIN32_STREAM_ID.

Cheers,
  FlatCap (Rich)
  nt...@fl...

Re: [Linux-NTFS-Dev] hidden files?

[farmer d. <far...@ya...>]

Morten

http://patriot.net/~carvdawg/perl.html

check out Astream.pl and ads.pl

www.foundstone.com

under their tools section check out sfind


SysInternals had streams available as well.

Lastly, lads is another tool.  Hope one of these prove
helpful for you.


farmerdude


--- morten <mo...@mo...> wrote:
> Hi 
> I am trying to recover some files from an xp
> machine, and It works nicely. I
> use the trinux boot disk and smbmount a share to
> copy every onto.
> 
> The problem is that I apparently haven't copied
> everything. A quick search on
> the net reveiled the concept og ads - alternate data
> streams - Supposedly to be
> a huge sevuroty hole. 
> My question is wheter ntfs have these very hidden
> files og directories. And how
> to access them.
> 
> Any input
> morten
> -- 
> ----------------------------------
> E-Mail: morten <mo...@mo...>
> Date: 19-Dec-2002
> Time: 00:41:26
> ----------------------------------
> 
> 
>
-------------------------------------------------------
> This SF.NET email is sponsored by: Order your
> Holiday Geek Presents Now!
> Green Lasers, Hip Geek T-Shirts, Remote Control
> Tanks, Caffeinated Soap,
> MP3 Players,  XBox Games,  Flying Saucers,  WebCams,
>  Smart Putty.
> T H I N K G E E K . C O M      
> http://www.thinkgeek.com/sf/
> _______________________________________________
> Linux-NTFS-Dev mailing list
> Lin...@li...
>
https://lists.sourceforge.net/lists/listinfo/linux-ntfs-dev


__________________________________________________
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
http://mailplus.yahoo.com

Re: [Linux-NTFS-Dev] hidden files?

[Richard R. <ri...@fl...>]

Hi Morten,

> I am trying to recover some files from an xp machine, and It works nicely. I
> use the trinux boot disk and smbmount a share to copy every onto.

Hmm, "trying", "recover", linux.

> The problem is that I apparently haven't copied everything.
> A quick search on the net revealed the concept of alternate data streams

OK, you are correct.  The Linux NTFS driver doesn't allow you to read
the other (named) data streams.

> My question is whether ntfs have these very hidden files or directories.

Yes NTFS has them, but fortunately, they aren't used often.  The only
common example I know of is to store summary information for files,
etc.  I don't think anyone would _rely_ on that information, though,
because it's so easily lost.  If you copy the file to a FAT partition,
e.g. a floppy.

I don't know if named data streams are preserved over windows file
sharing (between two windows machines).

> Supposed to be a huge security hole.

Hmm, I don't know about that.

> And how to access them.

I don't actually know how you list the data streams of a file (other
than reading the disk with a hex editor).

I hope this answers your questions,
  FlatCap (Rich)
  ri...@fl...

[Linux-NTFS-Dev] hidden files?

[morten <mo...@mo...>]

Hi 
I am trying to recover some files from an xp machine, and It works nicely. I
use the trinux boot disk and smbmount a share to copy every onto.

The problem is that I apparently haven't copied everything. A quick search on
the net reveiled the concept og ads - alternate data streams - Supposedly to be
a huge sevuroty hole. 
My question is wheter ntfs have these very hidden files og directories. And how
to access them.

Any input
morten
-- 
----------------------------------
E-Mail: morten <mo...@mo...>
Date: 19-Dec-2002
Time: 00:41:26
----------------------------------

[Linux-NTFS-Dev] [PATCH] ntfs_attrs_walk & AT_ATTRIBUTE_LIST

[Szakacsits S. <sz...@si...>]

Hi,

To do the cluster relocations for fragmented volumes, ntfsresize's
cluster accounting moved from low level functions to middle level ones
and caught a libntfs problem:  ntfs_attrs_walk (ntfs_attr_lookup)
doesn't enumerate attribute list attributes. Below is a patch. Fix
supposes AT_STANDARD_INFORMATION always present for a valid MFT record
if an AT_ATTRIBUTE_LIST is present. This should be always true (if not
ntfsresize catches it). All my testcases pass again.

	Szaka

--- libntfs/attrib.c	15 Dec 2002 13:02:00 -0000	1.4
+++ libntfs/attrib.c	18 Dec 2002 10:13:24 -0000
@@ -1743,7 +1743,10 @@
 		base_ni = ctx->base_ntfs_ino;
 	else
 		base_ni = ctx->ntfs_ino;
-	if (!base_ni || !NInoAttrList(base_ni) || type == AT_ATTRIBUTE_LIST)
+	/* FIXME: clean up *attr_find mess[y condition] */
+	if (!base_ni || !NInoAttrList(base_ni) || type == AT_ATTRIBUTE_LIST ||
+			(NInoAttrList(base_ni) && !type &&
+			ctx->attr->type == AT_STANDARD_INFORMATION))
 		return ntfs_attr_find(type, name, name_len, ic, val, val_len,
 				ctx);
 	return ntfs_external_attr_find(type, name, name_len, ic, lowest_vcn,

Re: [Linux-NTFS-Dev] NTFS-fs warning

[Richard R. <nt...@fl...>]

Hi Otto,

Sorry for the slow reply (it's the party season :-)

> ntfs_read_locked_inode(): BUG: Found compressed file with data_size not 
> equal to initialized_size. This will probably cause problems when trying 
> to access the file. Please notify lin...@li... that you 
> saw this message. Thanks!

Aha.  Don't worry we've seen this before and you shouldn't notice any.
problems.

Without official docs it's hard to predict what will and what won't
happen.  Therefore we've got lots of BUG lines in the code to check for
things.

> The system is a SuSE 8.1 with Kernel 2.4.19-4GB

Thanks for the bug report.
  FlatCap (Rich)
  nt...@fl...

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Leonard N. <lk...@ac...>]

I'm happy to report that with today's version, everything works! I'll 
give libntfs some other large drives to work on later to see what it 
says about them too (no files to rescue there, though ;-).

Many thanks, Szakacsits, Anton and Richard & co (who's been answering my 
questions on #ntfs)!

-- Leonard

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Anton A. <ai...@ca...>]

Comment removal done.

I also generated a new snapshot so it has all the fixes in it.

Anton

On Fri, 13 Dec 2002, Szakacsits Szabolcs wrote:

> 
> On Fri, 13 Dec 2002, Anton Altaparmakov wrote:
> 
> > Ok, I apologize. There is a bug in the library. It is just that your fix
> > is completely wrong.
> 
> Depends :) I had 3 solutions and my first one (not sent) was exactly
> (plus more) what you checked in and I explained why I sent what I've
> sent. However after your arguments I agree the current way seems to be
> the best and please apply this patch also. Thanks,
> 
> 	Szaka
> 
> --- ntfsprogs/ntfsresize.c	12 Dec 2002 08:31:38 -0000	1.1.1.1
> +++ ntfsprogs/ntfsresize.c	13 Dec 2002 16:57:36 -0000
> @@ -700,7 +700,6 @@
>  	if (write_mft_record(ctx))
>  		perr_exit("Couldn't update $BadClust");
> 
> -	/* FIXME: clean up API => ntfs_put_attr_search_ctx() also closes ni */
>  	ntfs_attr_put_search_ctx(ctx);
>  }
> 
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:
> With Great Power, Comes Great Responsibility 
> Learn to use your power at OSDN's High Performance Computing Channel
> http://hpc.devchannel.org/
> _______________________________________________
> Linux-NTFS-Dev mailing list
> Lin...@li...
> https://lists.sourceforge.net/lists/listinfo/linux-ntfs-dev
> 

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Anton A. <ai...@ca...>]

It does indeed. Applied. Thanks.

Anton

On Fri, 13 Dec 2002, Szakacsits Szabolcs wrote:

> On Fri, 13 Dec 2002, Anton Altaparmakov wrote:
> 
> > This is guaranteed. But now it doesn't matter any more. You are simply not
> > allowed to close extent inodes. If you do, you get what you deserve. But I
> > agree with the below patch. It adds a nice safety check. I will make it
> > different however so that it outputs rude error messages when that code
> > gets hit so people are told not to do this again... (-;
> 
> I'm afraid libntfs has two such examples itself ;)
> 
> 	Szaka
> 
> --- libntfs/attrib.c	13 Dec 2002 08:24:35 -0000	1.2
> +++ libntfs/attrib.c	13 Dec 2002 16:40:06 -0000
> @@ -1559,9 +1559,6 @@
>  				break;
>  			}
>  		} else { /* Mft references do not match. */
> -			/* If there is a mapped extent inode unmap it first. */
> -			if (ni != base_ni)
> -				ntfs_inode_close(ni);
>  			/* Do we want the base record back? */
>  			if (MREF_LE(al_entry->mft_reference) ==
>  					base_ni->mft_no) {
> @@ -1642,8 +1639,6 @@
>  		goto do_next_attr_loop;
>  	}
>  	if (ni != base_ni) {
> -		if (ni)
> -			ntfs_inode_close(ni);
>  		ctx->ntfs_ino = base_ni;
>  		ctx->mrec = ctx->base_mrec;
>  		ctx->attr = ctx->base_attr;
> 
> 
> -- 
> This email was forwarded via the University of Cambridge alumni email system
> Visit http://cantab.net/ to update your forwarding details
> 

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Szakacsits S. <sz...@si...>]

On Fri, 13 Dec 2002, Anton Altaparmakov wrote:

> Ok, I apologize. There is a bug in the library. It is just that your fix
> is completely wrong.

Depends :) I had 3 solutions and my first one (not sent) was exactly
(plus more) what you checked in and I explained why I sent what I've
sent. However after your arguments I agree the current way seems to be
the best and please apply this patch also. Thanks,

	Szaka

--- ntfsprogs/ntfsresize.c	12 Dec 2002 08:31:38 -0000	1.1.1.1
+++ ntfsprogs/ntfsresize.c	13 Dec 2002 16:57:36 -0000
@@ -700,7 +700,6 @@
 	if (write_mft_record(ctx))
 		perr_exit("Couldn't update $BadClust");

-	/* FIXME: clean up API => ntfs_put_attr_search_ctx() also closes ni */
 	ntfs_attr_put_search_ctx(ctx);
 }

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Szakacsits S. <sz...@si...>]

On Fri, 13 Dec 2002, Anton Altaparmakov wrote:

> This is guaranteed. But now it doesn't matter any more. You are simply not
> allowed to close extent inodes. If you do, you get what you deserve. But I
> agree with the below patch. It adds a nice safety check. I will make it
> different however so that it outputs rude error messages when that code
> gets hit so people are told not to do this again... (-;

I'm afraid libntfs has two such examples itself ;)

	Szaka

--- libntfs/attrib.c	13 Dec 2002 08:24:35 -0000	1.2
+++ libntfs/attrib.c	13 Dec 2002 16:40:06 -0000
@@ -1559,9 +1559,6 @@
 				break;
 			}
 		} else { /* Mft references do not match. */
-			/* If there is a mapped extent inode unmap it first. */
-			if (ni != base_ni)
-				ntfs_inode_close(ni);
 			/* Do we want the base record back? */
 			if (MREF_LE(al_entry->mft_reference) ==
 					base_ni->mft_no) {
@@ -1642,8 +1639,6 @@
 		goto do_next_attr_loop;
 	}
 	if (ni != base_ni) {
-		if (ni)
-			ntfs_inode_close(ni);
 		ctx->ntfs_ino = base_ni;
 		ctx->mrec = ctx->base_mrec;
 		ctx->attr = ctx->base_attr;

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Anton A. <ai...@ca...>]

On Fri, 13 Dec 2002, Szakacsits Szabolcs wrote:

> On Fri, 13 Dec 2002, Szakacsits Szabolcs wrote:
> > > > > 3. ntfs_extent_inode_open: extents are copied to a newly
> > > > >    allocated memory. Before using just the pointer equality the
> > > > >    memory was freed by the extent and the base inode happily used
> > > > >    the unfreed memory => severe memory corruptions.
> >
> > The fix for 3 was uncomplete [sent you update], it didn't copy the
> > mrec.
> 
> The below alternative patch is also ok *if* it's guaranteed an extent
> inode is always closed before its base inode closed (note, there are
> two users/pointers to the extent!). However this is not guaranteed.

This is guaranteed. But now it doesn't matter any more. You are simply not
allowed to close extent inodes. If you do, you get what you deserve. But I
agree with the below patch. It adds a nice safety check. I will make it
different however so that it outputs rude error messages when that code
gets hit so people are told not to do this again... (-;

Thanks.

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Anton A. <ai...@ca...>]

On Thu, 12 Dec 2002, Szakacsits Szabolcs wrote:
> On Thu, 12 Dec 2002, Anton Altaparmakov wrote:
> > applied, thanks. well spotted again. I had actually fixed this exactly
> > same bug ages ago in the NTFS kernel driver but I obviously had forgotten
> > to port the fix to the library. A very annoying oversight on my part!
> 
> BTW, I wanted to ask for long but actually how much code is shared
> between the user and kernel "drivers"? I'm afraid zero. I know it's
> not trivial but there are codes (filesystems) that has this very
> nice feature.

The code is simillar (especially in functionality) but there are varying
amounts of differences between kernel and library code. Differences are
based on completely different amounts of caching and lack thereof in the
kernel and library, respectively. Basically the kernel code's entry points
are the VFS and the page cache/MM code while the library entry points are
from applications which do completely different things so the driver and
the library are naturally very different. Further the header file includes
are different as the kernel gets things from include/linux while userspace
is not allowed to do so. So basically it becomes a much bigger nightmare
to share code between the two then to have them separate...

Also the driver is in the kernel so there is no way to keep the code
shared now. It would have been possible when the code was outside the
kernel but now it would just make a huge PITA.

> > > 3. ntfs_extent_inode_open: extents are copied to a newly
> > >    allocated memory. Before using just the pointer equality the
> > >    memory was freed by the extent and the base inode happily used
> > >    the unfreed memory => severe memory corruptions.
> >
> > I don't understand this description and I don't understand why you are
> > doing the copying.
> 
> You allocate an ntfs_inode for an extent then attach it *also* to the
> extents of it's base inode, 

Wrong. We don't *also* attach it to its base inode. We *only* attach it to
its base inode. Extent inodes are illegal and cannot exist/be open without
their base inode being open, too.

You cannot use the extent inode after you have used the base inode. That
is against the whole concept of extent inodes.

To use an extent inode you *must* do:

ntfs_inode_open(base inode);
ctx = ntfs_attr_get_search_ctx(base inode);
ntfs_attr_lookup(base inode, ctx);

make use of ctx->ntfs_ino (which can be base or extent inode, it doesn't
matter);

ntfs_attr_put_search_ctx(ctx); --> this destroys the extent inode
ntfs_inode_close(base_inode); --> this destroys the base inode

Any other use of extent inodes is illegal and not allowed.

> after you free ntfs_inode => the attached
> extents live in the freed memory and the next time malloc gives
> it away and will be nicely overwritten it.

I don't understand this. When the ntfs inode is closed, all attached
extent inodes are freed automatically. No memory is wasted.

I really don't understand where you see memory being leaked or corrupted.

> > corruptions? - Your fix 2. from above should cure everything. The memory
> 
> No, just use a memory debugger or printf the pointer.

I don't see that being necessary unless you can show me the lines of code
that cause the corruption. AFAICS the libntfs code is logically consistent
and does not leak memory anywhere.

I suspect that what you are talking about is simply incorrect use of the
library as opposed to a library bug and obviously this needs to be fixed
not in the library but at the point where it is used.

> > allocations and copying are already done in the if clause fixed by point
> > 2 so I don't see why you repeat them again here. What am I missing?
> 
> The if clause copies the extent pointers. Afterwards the ntfs_inode
> are copied to a newly allocated area (because the ntfs_inode in
> question will be freed by ntfs_inode_close but the base inode still
> needs it later on).

You never close extent inodes.

Ah, oops, we do close extent inodes when we do ntfs_attr_put_search_ctx().

Ok, I apologize. There is a bug in the library. It is just that your fix
is completely wrong. I will think about it and fix it properly. At initial
thought, the fix will simply be either to 1) remove the
ntfs_inode_close() call from ntfs_attr_put_search_ctx() and just let the
main call of ntfs_inode_close(base inode) get rid of all the extents, or
2) make ntfs_inode_close(extent inode) remove the inode from the base
inode's list of extent inodes. Either of 1) and 2) would work but I need
to think a little about which we actually want to have. I strongly suspect
we want 1) so we get the extents cached in memory at all times so we don't
have to keep re-reading them.

Ok, I have now done 1) and pushed to the bkbits repository. Also there is
now a new snapshot available in usualy place
(http://linux-ntfs.sf.net/snapshots).

Can people try and let me know if it fixes everything?

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Szakacsits S. <sz...@si...>]

On Fri, 13 Dec 2002, Szakacsits Szabolcs wrote:
> > > > 3. ntfs_extent_inode_open: extents are copied to a newly
> > > >    allocated memory. Before using just the pointer equality the
> > > >    memory was freed by the extent and the base inode happily used
> > > >    the unfreed memory => severe memory corruptions.
>
> The fix for 3 was uncomplete [sent you update], it didn't copy the
> mrec.

The below alternative patch is also ok *if* it's guaranteed an extent
inode is always closed before its base inode closed (note, there are
two users/pointers to the extent!). However this is not guaranteed.
Reference counting would solve this or the "private extent copies"
patches I sent you. Chose whatever you feel the most appropriate.

	Szaka

--- libntfs/inode.c	13 Dec 2002 12:28:52 -0000	1.3
+++ libntfs/inode.c	13 Dec 2002 15:48:52 -0000
@@ -202,6 +202,9 @@
 		errno = EBUSY;
 		return -1;
 	}
+	/* If mapped extent inode then base inode will free it */
+	if (ni->nr_extents = -1)
+		return 0;
 	/* Is this a base inode with mapped extent inodes? */
 	if (ni->nr_extents > 0) {
 		int i;

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Szakacsits S. <sz...@si...>]

On Thu, 12 Dec 2002, Szakacsits Szabolcs wrote:

> > > 3. ntfs_extent_inode_open: extents are copied to a newly
> > >    allocated memory. Before using just the pointer equality the
> > >    memory was freed by the extent and the base inode happily used
> > >    the unfreed memory => severe memory corruptions.
>
> > seems the patch gets me past the overlapping runlists problem but, I get
> > 11298 messages saying Couldn't read MFT record %n (where %n is the record
> > number, of course).
>
> On an MFT with 20+ extents I get one
> ------------------------------------------------------->
> Entering for inode 0, attribute type 0x80.
> Opening extent inode 15 (base mft record 0x0).
> Found stale extent mft reference! Corrupt file system. Run chkdsk.
> Failed to map extent inode: Input/output error
> Inode is corrupt.
> Couldn't read MFT Record 28.
> <--------------------------------------------------------
>
> Probably the above bugs hid more - so far Leonards case (attribute
> list handling) discovered 4 bugs.

The fix for 3 was uncomplete [sent you update], it didn't copy the
mrec. Now ntfsundelete (libntfs) works also with non-resident
attribute list attribute in $MFT.

	Szaka

[Linux-NTFS-Dev] =?ks_c_5601-1987?B?wNq68cPixscgvsizu1u9w8H9LLz2x8osvNK8syyw5sGmLLD=?= =?ks_c_5601-1987?B?5r+1LMOzvLwgseLFuF1bsaSw7V0=?=

[<h1...@hh...>]

v/iw7cHfsLMsIMDauvHD4sbHLCDA2rytwPwsILHiyLnD4sbHILXuIMPixse8rbrxvbogwPy5rsi4
u+cgIMDauvHD4sbHIL7Is7sovcPB/Sy89sfKLLzSvLMssObBpiyw5r+1LMOzvLwsseLFuCkgICAg
DQpsaW51eC1udGZzLWRldrTUsrIgIMfgurnAzCDH1LKyx8+9w7HiuKYguve0z7TZLg0Ksc3Hz8DH
ICC43sDPwda80rTCIMClvK3HzsHfv6G8rSC+y7DUILXIsM3AzLjnLCBFLU1haWwgwda80iC/3L+h
LCC02bilICDBpLq4tMIgsK6w7SDA1sH2IL7KvcC0z7TZLiC6uyC43sDPwLogwaS6uMXrvcW4wSDA
zL/rw8vB+CC51yDBpLq4urjIoyAgte6/oSCw/MfRILn9t/wgwaYgNTDBtr+hIMDHsMXH0VuxpLDt
XSC43sDPwNS0z7TZDQoNCiANCrz2vcWwxbrOuKYgv/jHz73DuOkgv6mx4rimIMWsuK/Hz7HiuLgg
x8+8vL/kfiANCiANCiANCiAgW8DauvHD4sbHwMcgIMDavLzH0SCzu7/rXQ0KICDA+sDbsce758fX
DQogIMD6wNrAxyAgu+e787D6IMO2x9DAuyC0477GIMewwKfA1rTCIMOlwLi3ziC4uLXpvu4gILXl
uLO0z7TZLiAgw+LGx7Xut8+w+iAgsbnBpiC5rsflufjIoyhJU0JOKbimILrOv6nH2CC15biztM+0
2S4gIA0KILG5uLPB377TtbW8rbD8v6EgILOzurvHz7+pIL7wwaa287W1IL+ttvfHz73HILz2IMDW
vcC0z7TZLiAgDQogwPrA27HHwLsgIMO2wPrHz7DUILq4wOXH1bTPtNkuICAgIMGmwNu55rn9DQog
ILPrt8PH0SAgv+6/tcC4t84gwabA27rxuKYgw9a068fRIMD9vuDHz7+pIMD6t8XH0SAguvG/68DM
ILXstM+02S4gDQogw7bA+sfPsNQgIMD6wNrAxyDAx7W1uKYgMTAwJSC53b+1x9W0z7TZLg0KILq7
ua6w+iAgx6XB9sDHILGzwaTB9rimIMPWvNLH0SC1ziC5+CDAzLvzILq4s7u15biztM+02S4gIA0K
IMD6wNrAxyAgT0u758DOwMwgs6q+37i4IMGmwNu/oSC16b7usKm0z7TZLiANCiCw+LDdwPvAziAg
sOa/tcC4t84gsMW3obytwaHAuyDHz7fntbUguvzB9sH2IL7KsO0gILnmua7Hz7+pILW1vK3AxyDB
+L+tu/PFwrimILz2vcO3ziDBobDLx8+w7SwgILT1v+0gwcHAuiDA2riuv6Egwfi/rbXJILz2IMDW
tbW3zyDD1ryxwLsgILTZx9W0z7TZLiAgICAgIMDOvLwgILDovuC55r3EDQogIMOzwL0gIMGmwNu6
8cDHIMDPus64piDA+sDasKEgus6048fPtMIgsM0gv9y/oSAgwM+53SDD4sbHubCw+iC1v8DPx9Eg
sOi+4CDD37ChuvG/6yAgwPzH9CC++MC4uOcgw6XAzCDA3yDGyLiuuOkgwPrA2r+hsNQgIMDOvLy4
piDB9rzTwPvAuLfOIMH2sd4gMTAlICC8scDOvLwgICANCsDavLzH0SAgu+fH18C6ILW1vK3D4sbH
IMDbwLq9o7+hvK0gILvsxuy6uL3DseIgudm2+LTPtNkuDQoNCiAgICAgtbW8rcPixscgwNvAur2j
IA0KIA0KDQogDQo=

Re: [Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Szakacsits S. <sz...@si...>]

On Thu, 12 Dec 2002, Anton Altaparmakov wrote:

> applied, thanks. well spotted again. I had actually fixed this exactly
> same bug ages ago in the NTFS kernel driver but I obviously had forgotten
> to port the fix to the library. A very annoying oversight on my part!

BTW, I wanted to ask for long but actually how much code is shared
between the user and kernel "drivers"? I'm afraid zero. I know it's
not trivial but there are codes (filesystems) that has this very
nice feature.

> > 3. ntfs_extent_inode_open: extents are copied to a newly
> >    allocated memory. Before using just the pointer equality the
> >    memory was freed by the extent and the base inode happily used
> >    the unfreed memory => severe memory corruptions.
>
> I don't understand this description and I don't understand why you are
> doing the copying.

You allocate an ntfs_inode for an extent then attach it *also* to the
extents of it's base inode, after you free ntfs_inode => the attached
extents live in the freed memory and the next time malloc gives
it away and will be nicely overwritten it.

> corruptions? - Your fix 2. from above should cure everything. The memory

No, just use a memory debugger or printf the pointer.

> allocations and copying are already done in the if clause fixed by point
> 2 so I don't see why you repeat them again here. What am I missing?

The if clause copies the extent pointers. Afterwards the ntfs_inode
are copied to a newly allocated area (because the ntfs_inode in
question will be freed by ntfs_inode_close but the base inode still
needs it later on).

> seems the patch gets me past the overlapping runlists problem but, I get
> 11298 messages saying Couldn't read MFT record %n (where %n is the record
> number, of course).

On an MFT with 20+ extents I get one
------------------------------------------------------->
Entering for inode 0, attribute type 0x80.
Opening extent inode 15 (base mft record 0x0).
Found stale extent mft reference! Corrupt file system. Run chkdsk.
Failed to map extent inode: Input/output error
Inode is corrupt.
Couldn't read MFT Record 28.
<--------------------------------------------------------

Probably the above bugs hid more - so far Leonards case (attribute
list handling) discovered 4 bugs.

	Szaka

[Linux-NTFS-Dev] Re: [PATCH] Re: Couldn't mount device: Input/output error

[Anton A. <ai...@ca...>]

On Thu, 12 Dec 2002, Szakacsits Szabolcs wrote:
> Hi,
> 
> Great report Leonard :) Apparently libntfs couldn't handle the
> situation when MFT had extents (very fragmented MFT). Attached patch
> should fix it.
> 
> Anton, the patch has 3 fixes, small but they are important:
> 
> 1. ntfs_external_attr_find don't loop more then it should

applied, thanks. well spotted.

> 2. ntfs_extent_inode_open: fix a negation typo causing
>    memory corruptions

applied, thanks. well spotted again. I had actually fixed this exactly
same bug ages ago in the NTFS kernel driver but I obviously had forgotten
to port the fix to the library. A very annoying oversight on my part!

> 3. ntfs_extent_inode_open: extents are copied to a newly
>    allocated memory. Before using just the pointer equality the
>    memory was freed by the extent and the base inode happily used
>    the unfreed memory => severe memory corruptions.

I don't understand this description and I don't understand why you are
doing the copying. What exactly is wrong and where are the memory
corruptions? - Your fix 2. from above should cure everything. The memory
allocations and copying are already done in the if clause fixed by point
2 so I don't see why you repeat them again here. What am I missing?

> I tested ntfsresize with the fix on several partitions and everything
> is ok (ntfsresize has severe cluster accounting/checks), except now
> the fragmented MFT case also works. Would you also try Leonard?

Cool. He just told me on irc:
---quote---
seems the patch gets me past the overlapping runlists problem but, I get 
11298 messages saying Couldn't read MFT record %n (where %n is the record
number, of course).
---eoq---

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

[Linux-NTFS-Dev] NTFS-fs warning

[<ott...@bu...>]

Hello folks,

I found the following lines in my syslog:

Dec 10 19:59:52 Triangle kernel: NTFS driver 2.1.0a [Flags: R/O MODULE].
Dec 10 19:59:52 Triangle kernel: NTFS volume version 1.2.
Dec 10 20:01:17 Triangle kernel: NTFS volume version 1.2.
Dec 10 22:08:54 Triangle kernel: NTFS-fs warning (device 03:02): 
ntfs_read_locked_inode(): BUG: Found compressed file with data_size not 
equal to initialized_size. This will probably cause problems when trying 
to access the file. Please notify lin...@li... that you 
saw this message. Thanks!
Dec 10 22:09:01 Triangle kernel: NTFS-fs warning (device 03:02): 
ntfs_read_locked_inode(): BUG: Found compressed file with data_size not 
equal to initialized_size. This will probably cause problems when trying 
to access the file. Please notify lin...@li... that you 
saw this message. Thanks!
Dec 10 22:30:22 Triangle kernel: NTFS-fs warning (device 08:05): 
ntfs_read_locked_inode(): BUG: Found compressed file with data_size not 
equal to initialized_size. This will probably cause problems when trying 
to access the file. Please notify lin...@li... that you 
saw this message. Thanks!
Dec 10 22:30:26 Triangle kernel: NTFS-fs warning (device 08:05): 
ntfs_read_locked_inode(): BUG: Found compressed file with data_size not 
equal to initialized_size. This will probably cause problems when trying 
to access the file. Please notify lin...@li... that you 
saw this message. Thanks!

The system is a SuSE 8.1 with Kernel 2.4.19-4GB

I hope you're not too torn about this ;-)

Otto

Re: [Linux-NTFS-Dev] Portability of NTFS userspace code

[Anton A. <ai...@ca...>]

On 10 Dec 2002, Richard Russon wrote:
> > You actually need to put the pragma inline,
> 
> True, but I don't think this is a problem for us.  We want most of our
> structures packed, but it doesn't matter if the other ones are too.
> This means we can either put a #prama in compat.h or simply compile with
> an equivalent option: /Zp1

Oh that is cool. I didn't know about the compiler option. That seems like
the best way. Can we do the detection in ./configure.in?

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-NTFS-Dev] I cant rm files from ntfs partition ...

[Anton A. <ai...@ca...>]

On Wed, 11 Dec 2002, Taras G. Prokopenko wrote:

> I have old version of ntfs driver(included in 2.4.20 tree).
> Ntfs partition was made by windows 2000(sp3).

Neither the old nor the new driver have file deletion implemented, hence
you can't use the rm command...

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cantab.net> (replace at with @)
Linux NTFS maintainer / IRC: #ntfs on irc.freenode.net
WWW: http://linux-ntfs.sf.net/ & http://www-stu.christs.cam.ac.uk/~aia21/

[Linux-NTFS-Dev] I cant rm files from ntfs partition ...

[Taras G. P. <tp...@dg...>]

I have old version of ntfs driver(included in 2.4.20 tree).
Ntfs partition was made by windows 2000(sp3).

/hope this will be useful

Re: [Linux-NTFS-Dev] New libntfs API

[Andrew C. <cl...@gn...>]

On Tue, Dec 10, 2002 at 01:43:43PM +0000, Anton Altaparmakov wrote:
> I have recently committed all (hopefully all) of the renames necessary to
> get a proper stable API for the ntfs library. Please everyone have a look
> through the library (header files would be sufficient to give an idea of
> what functions are exported) and let me know what you think and if there
> is anything that still needs renaming or changing before the final
> release.

It looks like a good start :)  OTOH, I'd like to see lots of
improvements:

* it should be possible for users of libntfs to pass their own IO
abstraction (rather than using the kernel's or file system's)

* error handling doesn't look to good :/

* progress meters?

* all the functionality of userland tools should be in libntfs, or
perhaps libntfs-mkfs, etc.  (This is already on your TODO :)


I think the first 3 items will change lots of code, and API entry
points.  Eg: in libparted, I implemented progress meters by having
all "big" functions accepting an extra parameter, "PedTimer* timer".

I think reiser4's (as opposed to reiser3's!) userland implementation
is fantastic... take a look!

	www.namesys.com/snapshots/2002.12.04

Is it possible to make libntfs look like that?

Is there any need to "stabilize" an API, or can you just freeze what
you have now, and work on a new, radically different API for later?

I think I'll have time to help with this stuff :)  (Uni vacation :)

Cheers,
Andrew