linux-ntfs-dev — Development discussions and bug reports.

[Linux-NTFS-Dev] Open Season

[Richard R. <nt...@fl...>]

Hi,

Anton and I (FlatCap) are busy working on improving the NTFS driver for
Linux.  We've copied our emails to this list to let you know what we're
up to.  We hope to keep this up to keep you informed and hopefully get
you involved.

They're open letters, so if you have any ideas, wisdom or just a point
of view that you'd like to share, please reply to them.  We'd prefer if
you replied to the mailing list, but any feedback (even privately) would
be really appreciated.

Cheers,
  FlatCap (Richard)
  nt...@fl...

Re: [Linux-NTFS-Dev] Repost: NTFS Writing - Analysis

[Richard R. <fl...@fl...>]

Hi Anton,

> Anton wrote...
> > Flatcap wrote...
> > The simplest example is touch_file().  We have to update the $MFT entry
> > and the Index Entry. We also need to touch the $MFT, to show that it has 
> > been altered.
>
> Does NT really update the times of the MFT record for $MFT every single 
> time the MFT is written to?!? It seems crazy... You could say this is 
> putting some very heavy wear on the disk sectors containing the first mft 
> record...
> 
> > Indeed if we copy NT exactly, then after every file READ, we need to 
> > update the access times of the file, i.e. The $MFT and its mirror will 
> > need updating.
> 
> Are you sure about this? AFAIK, NT will only update access times after a 
> time delta has passed since the last update.

Well the MFT times seem to reflect the time of the last update (and I suppose
I ought to prefix every sentence with "I think that")  The time delta sounds
like NT's fs cache.  My ill-informed guess is that many of the updates are
getting swallowed.

> > So, touch_file() looks like:
> > 
> >     touch_file()
> >         update_file()            # update file
> >         finalise()               # update $MFT
> > 
> >     update_file()
> >         Update $MFT entry        # update file's entry
> >         Update Index entry
> > 
> >     finalise()
> >         update_file ($MFT)       # update timestamp
> >         Copy $MFT to $MFTMirr
> > 
> > The finalise will be needed after every update.
> 
> I agree in general. However, I disagree about finalise(). IMO 
> update_file($MFT) and "Copy $MFT to $MFTMirr" should be one and the same 
> thing. - When we are writing to $MFT anywhere in offset 0 to 4 * MFT record 
> size, we have to write the data to $MFTMirr as well.

Yeah, fair enough for the practical details.  I just wanted to break down
the actions into separate disk writes.

> The best solution is 
> IMO to put this into the low level write routines themselves, in fact the 
> lower we go the better for transparency of $MFTMirr updates. After all the 
> update of $MFTMirr should be in the same transaction as the update of $MFT. 
> - Since the $MFTMirr is a byte-wise copy of $MFT's first four mft records, 
> rather than a functional copy, it is not a problem to have the copy done 
> real deep inside the write routines. - Maybe in (the yet to be written) 
> ntfs_write_page() or something similar.

Agreed.  Admittedly the low levels would just be dealing with clusters
and would know little of MFT and Mirror, but this is an important exception.

> > The actual touch is broken into update & finalise to prevent infintite
> > loops when touching the $MFT.
> 
> Infinite loops are something we definitely have to watch for carefully 
> during the driver design/implementation as NTFS is full of dangers 
> involving them due to having all the metadata being files...

I started writing a test prog, just to get some of the layout right.
It's a crummy program, but I found it easier to hack code than design :-)
(flatcap/text.c)

> > Another simple case is append_file().
> > 
> >     append_file()
> >         write_data()             # write data to disk
> >         touch_file()             # update timestamp
> > 
> >     write_data()
> >         Write data               # put data on disk
> >         Update $Bitmap           # set bits
> >         touch_file($Bitmap)      # update timestamp
> 
> Update $Bitmap AFTER writing the data?!? I think I am hearing voices and 
> they are all screaming "RACE condition"...

Damn, you're right :-)  I thought about the order *I'd* want things written.
I thought: Number one - commit the data to disk.  That way if things go wrong
something could possibly be recovered.  Then the bitmap.  Does sound a bit
dangerous, now you mention it.

> My suggestion would be:
>          append_file()...
>          write_attribute()...
>          resize_attribute()...
>          allocate_clusters()...

That looks good; I suppose my example was too simple.  I need to rework it
and think about attributes.

> > The code we have tries to do everything itself.  If we can break it down
> > into packets of work, two things happen.
> > We can mimic, possibly even USE the log file,
> 
> Yes.
> 
> > and second we could queue and coalesce similar requests.
> 
> No. This already happens at two levels and we really do not need to do it 
> and in fact can't do it usefully AFAICS. 1) At the level of the block 
> devices in ll_rw_block and friends consecutive accesses are already 
> batched/coalesced. 2) The page cache buffers multiple reads/writes and only 
> invokes NTFS when we either need to really do a write or read (basically 
> the stuff in address space operations). In the mean time we are never 
> invoked and we don't care... The VFS rules. (-;

OK.  I didn't want to rewrite the VFS, though I had a sort-of plan.
If we were queueing our requests, then when we encountered a problem
(bad sector, etc) we could try and work around it (or just rollback).
I need to start acquainting myself with reiserfs and ext3.
If journalling filesystems aren't doing their own queueing, then obviously
there's no reason for us to (and forget the LogFile).

FlatCap (Rich)
nt...@fl...

Re: [Linux-NTFS-Dev] 60GB NTFS vol: ls = "1 is not an index record"

[Anton A. <ai...@ca...>]

Dear Martin,

At 21:57 05/07/2001, Martin Rex wrote:
>Now I have one machine with a 60GByte NTFS partition
>(4096 byte clusters) and I seem to have a problem.
>The mount succeeds but when trying to list the (top level)
>directory I only get to see the error message
>"1 is not an index record" on the console screen.  :-(
>
>Is the reason for this problem known?  Is there a fix
>available (maybe in one of the newer 2.4.x Kernels)?

Very likely the reason is that you are overflowing one of the variables in 
the driver.

If this is the problem, it is fixed in recent kernels but only in the -ac 
series. So I would suggest you get kernel 2.4.6 and the latest -ac patch 
(at the moment it is 2.4.6-ac1). (At your option you can use 2.4.5 kernel 
with any of the -ac patches for 2.4.5, as NTFS driver hasn't changed since 
then.)

If that does not fix it, please email us again...

Best regards,

         Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

[Linux-NTFS-Dev] 60GB NTFS vol: ls = "1 is not an index record"

[Martin R. <Mar...@sa...>]

Dear NTFS developers,

I have successfully been using the ntfs filesystem driver
(in the linux-2.2.19 kernel) to mount (read-only for safety)
and access various of my NTFS partitions on several PCs,
up to 24 GBytes without problems.

Now I have one machine with a 60GByte NTFS partition
(4096 byte clusters) and I seem to have a problem.
The mount succeeds but when trying to list the (top level)
directory I only get to see the error message
"1 is not an index record" on the console screen.  :-(

Is the reason for this problem known?  Is there a fix
available (maybe in one of the newer 2.4.x Kernels)?
If you need any further infos/details feel free
to ask.  I'm not afraid of hacking, but I have
no real linux kernel hacking experience so far. 

(I'm not subscribed to the linux-ntfs-dev mailing list)

Thanks,
-Martin

Re: [Linux-NTFS-Dev] Repost: NTFS Writing - Analysis

[Anton A. <ai...@ca...>]

At 13:55 04/07/01, Richard Russon wrote:
>I've taken a careful look at all the actions we need to perform to write
>to an NTFS volume.  I've tried to break everything down into atomic
>pieces.
>
>The simplest example is touch_file().  We have to update the $MFT entry
>and the Index Entry. We also need to touch the $MFT, to show that it has 
>been altered.

Does NT really update the times of the MFT record for $MFT every single 
time the MFT is written to?!? It seems crazy... You could say this is 
putting some very heavy wear on the disk sectors containing the first mft 
record...

>So, touch_file() looks like:
>
>     touch_file()
>         update_file()            # update file
>         finalise()               # update $MFT
>
>     update_file()
>         Update $MFT entry        # update file's entry
>         Update Index entry
>
>     finalise()
>         update_file ($MFT)       # update timestamp
>         Copy $MFT to $MFTMirr
>
>The finalise will be needed after every update.

Apart from the nomenclature which I don't like (but that's details...) I 
agree in general. However, I disagree about finalise(). IMO 
update_file($MFT) and "Copy $MFT to $MFTMirr" should be one and the same 
thing. - When we are writing to $MFT anywhere in offset 0 to 4 * MFT record 
size, we have to write the data to $MFTMirr as well. - The best solution is 
IMO to put this into the low level write routines themselves, in fact the 
lower we go the better for transparency of $MFTMirr updates. After all the 
update of $MFTMirr should be in the same transaction as the update of $MFT. 
- Since the $MFTMirr is a byte-wise copy of $MFT's first four mft records, 
rather than a functional copy, it is not a problem to have the copy done 
real deep inside the write routines. - Maybe in (the yet to be written) 
ntfs_write_page() or something similar.

>Indeed if we copy NT exactly, then after every file READ, we need to 
>update the access times of the file, i.e. The $MFT and its mirror will 
>need updating.

Are you sure about this? AFAIK, NT will only update access times after a 
time delta has passed since the last update (I do not know how large the 
time delta is admittedly), and even then, only if update time updates are 
enabled in the registry (which they are by default, but setting the right 
registry key will deactivate the updates). - We will of course have a 
(re)mount option to disable access time updates and hence beet NT since it 
will per mounted volume and not for the whole driver. (-;

>The actual touch is broken into update & finalise to prevent infintite
>loops when touching the $MFT.

Infinite loops are something we definitely have to watch for carefully 
during the driver design/implementation as NTFS is full of dangers 
involving them due to having all the metadata being files...

>Another simple case is append_file().
>
>     append_file()
>         write_data()             # write data to disk
>         touch_file()             # update timestamp
>
>     write_data()
>         Write data               # put data on disk
>         Update $Bitmap           # set bits
>         touch_file($Bitmap)      # update timestamp

Update $Bitmap AFTER writing the data?!? I think I am hearing voices and 
they are all screaming "RACE condition"...

My suggestion would be:

         append_file()
         {
                 if (!write_attribute($DATA,...))
                         fail();
                 touch_file();
                 touch_file($MFT);
         }

         write_attribute()
         {
                 ... sees that it has to extend ...
                 if (!resize_attribute())
                         fail();
                 if (!inline: write_data_into_attribute &
                                 update attribute record &
                                 lock_inode_and_mft_record while doing this) {
                         resize_attribute(back to old size);
                         fail();
                 }
         }

         resize_attribute()
         {
                 inline: calculate nr_clusters needed;
                 if (!allocate_clusters())
                         fail();
                 lock_both_inode_and_mft_record_for_write(separate?);
                 if (!do_page_cache_magic_to_add_data_to_pages()) {
                         unlock_inode_and_mft_record();
                         deallocate_clusters();
                         fail();
                 }
                 inline: update_attribute record (run list, etc)
                 unlock_both_inode_and_mft_record(separate?);
         }

         allocate_clusters()
         {
                 lock_bitmap();
                 inline: allocate_clusters_and_create_run_list_of_them;
                 unlock_bitmap();
                 touch_file(bitmap);
                 return run_list;
         }

Note: I haven't written above optimized, it's just conceptual. Otherwise 
for example allocate clusters would not necessarily do the locking this way...

>The code we have tries to do everything itself.  If we can break it down
>into packets of work, two things happen.

>We can mimic, possibly even USE the log file,

Yes.

>and second we could queue and coalesce similar requests.

No. This already happens at two levels and we really do not need to do it 
and in fact can't do it usefully AFAICS. 1) At the level of the block 
devices in ll_rw_block and friends consecutive accesses are already 
batched/coalesced. 2) The page cache buffers multiple reads/writes and only 
invokes NTFS when we either need to really do a write or read (basically 
the stuff in address space operations). In the mean time we are never 
invoked and we don't care... The VFS rules. (-;

>Any thoughts / comments?

As above. Sorry it took so long to reply but I have been very busy in the 
lab...

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

[Linux-NTFS-Dev] Repost: NTFS Writing - Analysis

[Richard R. <fl...@fl...>]

Hi Everyone,

I've taken a careful look at all the actions we need to perform to write
to an NTFS volume.  I've tried to break everything down into atomic
pieces.

The simplest example is touch_file().  We have to update the $MFT entry
and the Index Entry.  We also need to touch the $MFT, to show that it has
been altered.  So, touch_file() looks like:

    touch_file()
        update_file()            # update file
        finalise()               # update $MFT

    update_file()
        Update $MFT entry        # update file's entry
        Update Index entry

    finalise()
        update_file ($MFT)       # update timestamp
        Copy $MFT to $MFTMirr

The finalise will be needed after every update.  Indeed if we copy NT
exactly, then after every file READ, we need to update the access times
of the file, i.e. The $MFT and its mirror will need updating.

The actual touch is broken into update & finalise to prevent infintite
loops when touching the $MFT.

Another simple case is append_file().

    append_file()
        write_data()             # write data to disk
        touch_file()             # update timestamp

    write_data()
        Write data               # put data on disk
        Update $Bitmap           # set bits
        touch_file($Bitmap)      # update timestamp

The code we have tries to do everything itself.  If we can break it down
into packets of work, two things happen.  We can mimic, possibly even
USE the log file, and second we could queue and coalesce similar requests.

Any thoughts / comments?

Cheers,
  FlatCap (Rich).
  nt...@fl...

[Linux-NTFS-Dev] NTFS driver TNG

[Anton A. <ai...@ca...>]

Hello all,

Flatcap (Richard) and I have decided it's about time we started using the 
mailing list for NTFS discussions instead of messaging each other by ICQ. 
Here is a quick summary of previous discussion about NTFS driver TNG module 
you might have noticed in CVS (currently mostly empty), it clarifies what 
TNG is and what the plans for development are. - Just thought people might 
be interested:

Anton (a while ago): "TNG = the next generation. seems to be a commonly 
used abbreviation for starting something from scratch. - So it's for a new 
driver code base. - I started it a while ago but it is now terribly out of 
date so I have to update it before I can commit it. This will be the 
everything in page cache all singing and dancing driver one day in the future."

Anton (last night): "I am starting to get thoroughly fed up with trying to 
fix the current driver. So much is not implemented and what is implemented 
has bugs all over the write code paths as I am finding out slowly and/or is 
so different from how I would like to have it that I am starting to 
seriously consider ditching the existing driver and starting on TNG. Or at 
the very least throwing out the write support altogether. It is very messed 
up. Just fixing cluster allocation took me almost half a week when it would 
have taken half an hour to write from scratch given the correct 
framework... I will think about it some more and will have a close look at 
the page cache over the weekend. I am gaining a better understanding of it 
slowly and I have somewhat of an idea of how to use it for NTFS (it's not 
going to be pretty but it should fly speedwise and I hope to be able to 
hide the nastiness within the NTFS address_space_operations, so the rest of 
the driver will be clean [I wish...])."

Flatcap: "Before I saw the 'TNG' dir, I thought of putting together a 
skeleton driver, just stubs of code.  Something upon which we could write a 
driver."

Anton: "The general idea is to write a read-only driver which will use the 
page cache and several slab caches for storage of structures but at the 
same time all SMP locking needed for writing is put in place. So to enable 
writing we can just add more functions, hopefully having done a good enough 
job of the reading and locking, so that then existing read code needs not 
be modified. Writing would then only happen into memory (page cache, slab 
cache, etc) in the normal code and writing to disk will only happen through 
kntfsd running as a kernel thread. When necessary we can force-wakeup 
kntfsd, possibly telling it somehow that something specific needs writing 
to disk. But I want to try and keep writing to device localized to kntfsd 
and nowhere else so that we have better control over whether writing is 
enabled or not. This will be extremely important for transactional writing 
and the log file system should we ever manage to implement it... - Once I 
have cleaned up TNG I will commit it. It will be pretty much a skeleton 
driver as I plan to ditch most of the stuff that is in it at the moment..."

Flatcap: "OK the logfile stuff may (always) be beyond our control, but 
there are very good reasons for transaction stuff.  I have a plan too!"

Anton: "Re: transaction stuff
Indeed. the simplest being that you can back trace to leave in a consistent 
state on error. (e.g. if in the middle of extending a file we fail, we want 
the file to return to it's state before the error rather than leaving a 
corrupt file on disk...)"

That's where we left off last night (note that this is a summary only)...

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

[Linux-NTFS-Dev] Windows Dynamic Disks -- I need your help

[Richard R. <nt...@fl...>]

Hi All,

I've just released v0.0.2 of Linux LDM (support for Windows Dynamic Disks)
In the package there's a kernel driver and also a test tool.  It can be
downloaded from the NTFS home page at:

  http://linux-ntfs.sourceforge.net

There's some basic documentation for the LDM at:

  http://linux-ntfs.sourceforge.net/ldm

Microsoft hasn't documented the Logical Disk Manager, so all this has
been written from scratch.  Despite that, the driver works.
It works for me and it works for a friend.

I need your help to test the code on other machines.
The test program is read-only and should be quite safe.

To build the tool just run make.  You should end up with a tool called
'drive' in the 'test' directory.  If it doesn't build please let me know.

For comparison:

If you run fdisk on a Windows Dynamic Disk, you'll see something like:

   Device Boot    Start     End  Blocks   Id  System
/dev/hda1            64   41328   20632+  42  SFS

All the managed partitions are hiding in this SFS partition.
It works a little bit like an extended partition.

Now, if you run 'drive' on the same disk, you'll see:

# test/drive /dev/hdb
Partition check:
 hdb: hdb1 < hdb5 hdb6 hdb7 hdb8 >

Device | Offset Bytes    Sectors     Mb | Size   Bytes    Sectors     Mb
-------+--------------------------------+-------------------------------
hdb    |            0          0      0 |     21159936      41328     20
hdb1   |     20111360      39280     19 |      1048576       2048      1
hdb5   |        32256         63      0 |      5242880      10240      5
hdb6   |      5275136      10303      5 |      5242880      10240      5
hdb7   |     10518016      20543     10 |      5242880      10240      5
hdb8   |     15760896      30783     15 |      4194304       8192      4

This shows the layout of the disk.  Partition 1 is the dataabase which is
stored at the end of the disk.  Partitions 5-8 are real data partitions.

(N.B. The device will always be hdb*)

There are two flags that can be used with test:
  --debug   Display lots information about what it's doing
  --dump    Save the database information to a file, e.g. hda.dump

If the information doesn't look correct, please email me the output from

  test --debug /dev/<device>


If you're feeling adventurous, you could try out the kernel driver.
The tool uses the same code, so if it works, you may be OK.  Copy
the source from the linux/ tree provided into your kernel source
tree and give it a go.  This is not a job for a kernel novice.

Any feedback would be great.
  I want to know if things go well
  I want to know if it breaks and where.

Many thanks,
  FlatCap (Richard)
  nt...@fl...

[Linux-NTFS-Dev] Announce: Linux support for Windows Dynamic Disks

[Richard R. <nt...@fl...>]

This is the first release of the Linux-LDM package.
It is available at:

  http://sf.net/project/showfiles.php?group_id=13956

Documentation, also very rough, is available at:

  http://linux-ntfs.sf.net/ldm

The package contains:

 * A simple tool to dump the LDM information
 * A Linux kernel driver to partition LDM volumes.

The driver works, but needs lots more testing.
The tool is primarily to test the driver.

To build and use the kernel driver you will need a small amount
of kernel knowledge.  It's also important to note what the kernel
driver can and cannot do.

 * It can divide the disk into partitions.
 * It cannot understand stripes, mirrors, spans or raid.

Information about Windows Dynamic Disks is stored in a database
at the end of the physical disk.  The parsing of the database
is just the first step.  Next, we need a Volume Manager to build
the partitions into usable volumes.

Any volunteers?

Any questions, comments or problems, please let me know.

Cheers,
    FlatCap (Richard)
    nt...@fl...

[Linux-NTFS-Dev] ANN: Linux-NTFS 1.0.0 stable released

[Anton A. <ai...@ca...>]

Linux-NTFS 1.0.0 stable is now released.

Get it from http://sf.net/projects/linux-ntfs/

NTFS utilities
==============
NtfsFix v1.15 - Attempt to fix an NTFS partition that has been damaged by 
the Linux NTFS driver. Note that you should run it every time after you 
have used the Linux NTFS driver to write to an NTFS partition to prevent 
massive data corruption from happening when Windows mounts the partition. 
IMPORTANT: Run this only *after* unmounting the partition in Linux but 
*before* rebooting into Windows NT/2000 or you *will* suffer! - You have 
been warned!

mkntfs v1.39 - Format a partition with the NTFS filesystem. See man 8 
mkntfs for command line options.

NtfsDump_LogFile v1.0 - Interpret and display information about the journal 
($LogFile) of an NTFS volume.

dumplog v1.0. - As NtfsDump_LogFile but operates on a file rather than a 
partition, so can use it on a live mounted file system.

ldm v1.3 - Interpret and display the Logical Disk Manager database (of a 
Windows 2000/XP dynamic disk/block device).

NTFS library
============
Provides common NTFS access functions to the ntfstools and other foreign 
open source applications. Note, that the library is still under heavy 
development and doesn't include the majority of functionality yet. It only 
is capable of just about supporting the current ntfstools, so I wouldn't 
recommend using it for your own applications at this stage.

Changes:
* mkntfs release and bugfixes to libntfs and the other utilities. Includes 
a man (8) page.
* ldm release which dumps the ldm database on Win2k/XP dynamic disks.
* updated ntfsdump_logfile and new dumplog operating on the logfile itself 
rather than the partition, suitable for live mounted file systems.
* Building of shared libraries is disabled by default as it breaks on some 
systems.
* Probably need at least gcc-2.95 or something like that from now on.

Please report any bugs/problems to:
         lin...@li...

Enjoy,

         Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://sf.net/projects/linux-ntfs/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

[Linux-NTFS-Dev] Attention DISC users and distributors!

[<st...@DI...>]

<html>

<head>
<meta http-equiv="Content-Language" content="en-us">
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<meta name="GENERATOR" content="Microsoft FrontPage 4.0">
<meta name="ProgId" content="FrontPage.Editor.Document">
<title>Attention DISC Users!</title>
</head>

<body>

<table border="0" width="100%" cellspacing="0" cellpadding="0">
  <tr>
    <td width="1145" height="21" colspan="2"><img border="0" src="http://www.discinsights.com/webmole/discusers/images/topBar.gif" width="500" height="122"></td>
  </tr>
  <tr>
    <td width="235" valign="top"><img border="0" src="http://www.discinsights.com/webmole/discusers/images/sideGuy.gif" width="235" height="282">
      <p align="center"><b><font face="Arial" size="2" color="#C01000"><i>Visit</i></font></b><font face="Arial" size="2"><a href="http://www.personalitystyle.com"><br>
      </a><a href="http://www.discinsights.com/Profile24XR/index.cfm?AffiliateID=30">
      <img border="0" src="http://www.discinsights.com/webmole/discusers/images/psLogo.gif" width="200" height="70"></a></font><b><i><font face="Arial" size="2"><br>
      <font color="#000000">for your free personality report!</font></font></i></b></p>
      <p align="center"><b><i><font face="Arial" size="2"><font color="#C01000">Or
      visit:</font><br>
      </font></i></b><a href="http://www.discinsights.com"><img border="0" src="http://www.discinsights.com/webmole/discusers/images/lgoDISCinsights.gif" width="100" height="64"></a><br>
      <b><i><font face="Arial" size="2" color="#000000">For our on-line catalog!</font></i></b></p>
      <p align="center">&nbsp;</td>
    <td valign="top" width="908">
      <p align="center"><font face="Arial"><b><i>Paper or electronic...</i></b></font></p>
      <p align="center"><b><font size="5" face="Arial" color="#FF0000">pay less</font><font color="#000000"><font face="Arial"> and
      </font></font><font size="5" face="Arial" color="#FF0000"> get more</font><font face="Arial" color="#000000"> from your DISC personality profiles.</font></b></p>
      <p align="left"><b><font face="Arial"><i>We offer...</i></font></b></p>
      <ul>
        <li>
          <p align="left"><font face="Arial" size="2">  the lowest-priced profiles on the market</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">self-scored paper profiles</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">automatically-scored electronic profiles</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">accurate and comprehensive assessments</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">instant online reports users can print immediately</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">professional-quality online reports</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">a user-friendly, simple online system</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">customized covers by request</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">an affiliate program (paper/electronic) available as profit center for distributors</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">a marketing starter kit for distributors</font></li>
      </ul>
      <p align="left"><b><font face="Arial"><i>Need more information?</i></font></b></p>
      <ul>
        <li>
          <p align="left"><font face="Arial" size="2">Visit <a href="http://www.discinsights.com/Profile24XR/index.cfm?AffiliateID=30">http://www.personalitystyle.com</a> and receive a free profile report.</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">Call <b>1-800-779-DISC </b>(3472) and request a free printed sample of the full profile report.&nbsp;</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">Visit <a href="http://www.discinsights.com/Profile24XR/index.cfm?AffiliateID=30">http://www.personalitystyle.com</a> to learn more about our affiliate program and how you may become a distributor.</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">Visit <a href="http://www.discinsights.com">http://www.discinsights.com</a> to view our complete online catalog of DISC-related assessment instruments.</font></li>
        <li>
          <p align="left"><font face="Arial" size="2">Reply to this email if you would like to have a representative contact you by phone.  Please leave your name and phone number.</font></li>
      </ul>
          <p align="left">&nbsp;
    </td>
  </tr>
</table>

</body>

</html>

[Linux-NTFS-Dev] Announce: Win2K LDM Docs (Logical Disk Manager)

[Richard R. <nt...@fl...>]

Hi All,

I'm pleased to announce the first version of the LDM Documentation.

Windows 2000 introduced a new partitioning scheme and with it, the
Logical Disk Manager.  Like Linux's Logical Volume Manager is allows
changes to partitioning, and volumes, to be made without rebooting.
To create Mirrored, Spanned, Striped or RAID disks under Win2K, you
must use their "Dynamic Disks".

Unfortunately Linux cannot read these Dynamic Disks.  Yet.

The documentation, although only a first draft, contains enough
information to locate and piece together the disks, partitions and
volumes used by Win2K.  The docs show the on-disk structures that
make up the 1MB database at the end of the physical disk.

The docs are available online at:

  http://linux-ntfs.sourceforge.net/ldm

and to download at:

  http://sourceforge.net/project/showfiles.php?group_id=13956

Also, I've written a test program to dump the LDM information stored
on a dynamic disk.  It's available at:

  http://linux-ntfs.sourceforge.net/ldm/ldm.c

If you have any questions, comments or additions, please email me.

Thanks,
  FlatCap (Richard Russon)
  nt...@fl...

[Linux-NTFS-Dev] REQ: Anyone have WinNT 3.51 partition image?

[Anton A. <ai...@ca...>]

Hi,

If anyone has a small partition image they could give me that was formatted 
with WinNT 3.51 (and hasn't been upgraded by a later version of 
WindowsNT/2k/XP) it would be great if you could let me know and if you 
don't mind we can arrange for sending it to me.

Alternatively, if you have a large WinNT 3.51 partition so that sending is 
not an option but you would be willing to run a small diagnostic Linux 
program on it, it would also give me the info I want to find, (I will 
supply source and/or static linked binary, whatever you prefer, note: NTFS 
fs driver will not be needed for this, the utility will read the disk 
device directly).

Thanks a lot in advance!

Best regards,

         Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://sf.net/projects/linux-ntfs/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Jeff V. M. <jm...@vg...>]

On Tue, May 29, 2001 at 10:18:41AM +0200, Martin von Loewis wrote:
> > > Anyone know about 3.1?
> > > 
> > 
> > It's an HPFS variant.
> 
> No. NT was using NTFS right from the start.
> 
> Regards,
> Martin

No.  They were not.  Their first cuts of NT used an HPFS variant until 
NTFS could be completed.  The guy who wrote NTFS for MS was a consultant
for us for several years, and I got a very long briefing on the history
of NT file systems over the years.

I do not know which version was the cutover, but I do believe it was 
pre 3.X NT versions.  

Jeff

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Martin v. L. <lo...@in...>]

> > Anyone know about 3.1?
> > 
> 
> It's an HPFS variant.

No. NT was using NTFS right from the start.

Regards,
Martin

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Jeff V. M. <jm...@vg...>]

On Mon, May 28, 2001 at 03:49:28PM +0100, Anton Altaparmakov wrote:
> At 14:08 28/05/2001, Yuri Per wrote:
> >Anton Altaparmakov wrote:
> >
> >>Does anyone know what NTFS version the NT 3.1 / 3.51 volumes had? If I 
> >>know  I can make sure we don't mount such beasts considering we know the 
> >>driver  would fail on them... - I am aware of only one person stil using 
> >>NT 3.51  and he doesn't believe in the NTFS Linux driver any more, so I 
> >>guess we can  just say we support NT 4.0 and above only.
> >
> >NT 3.51 uses exactly the same version of NTFS as NT 4.0 does.
> 
> Ok. Thanks.
> 
> Anyone know about 3.1?
> 
> Anton
> 

It's an HPFS variant.  Try the HPFS driver, and it may work.  The first cut
of NT was with a hacked up HPFS driver from Microsoft OS/2.  NTFS was 
designed internally by David G. and friends.

Jeff

> 
> -- 
>    "Nothing succeeds like success." - Alexandre Dumas
> -- 
> Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
> Linux NTFS Maintainer / WWW: http://sf.net/projects/linux-ntfs/
> ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to maj...@vg...
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Anton A. <ai...@ca...>]

At 14:08 28/05/2001, Yuri Per wrote:
>Anton Altaparmakov wrote:
>
>>Does anyone know what NTFS version the NT 3.1 / 3.51 volumes had? If I 
>>know  I can make sure we don't mount such beasts considering we know the 
>>driver  would fail on them... - I am aware of only one person stil using 
>>NT 3.51  and he doesn't believe in the NTFS Linux driver any more, so I 
>>guess we can  just say we support NT 4.0 and above only.
>
>NT 3.51 uses exactly the same version of NTFS as NT 4.0 does.

Ok. Thanks.

Anyone know about 3.1?

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://sf.net/projects/linux-ntfs/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Yuri P. <yu...@ac...>]

Anton Altaparmakov wrote:

> Does anyone know what NTFS version the NT 3.1 / 3.51 volumes had? If I 
> know  I can make sure we don't mount such beasts considering we know 
> the driver  would fail on them... - I am aware of only one person stil 
> using NT 3.51  and he doesn't believe in the NTFS Linux driver any 
> more, so I guess we can  just say we support NT 4.0 and above only.

NT 3.51 uses exactly the same version of NTFS as NT 4.0 does.


Yuri

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Anton A. <ai...@ca...>]

At 12:10 28/05/2001, Martin von Loewis wrote:
> > >That would not work: NT would split individual runs across extends
> > >(i.e. split them in the middle). Did I misunderstand, or do you have a
> > >solution for that as well.
> > >
> > Are you sure that it's true? My NTFS resizer interprets parts of runlist
> > stored in different FILE records independently and I never experienced
> > any problems with that.
>
>I'm sure it could happen on NT 3.1. Maybe MS has fixed it since.

Hm, in that case I think I will implement it the way I was thinking and I 
will assume that it can't happen any more. - I will add some kind of sanity 
check at the beginning to catch problems and scream to the syslog which we 
can remove later.

Does anyone know what NTFS version the NT 3.1 / 3.51 volumes had? If I know 
I can make sure we don't mount such beasts considering we know the driver 
would fail on them... - I am aware of only one person stil using NT 3.51 
and he doesn't believe in the NTFS Linux driver any more, so I guess we can 
just say we support NT 4.0 and above only.

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://sf.net/projects/linux-ntfs/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Martin v. L. <lo...@in...>]

> >That would not work: NT would split individual runs across extends
> >(i.e. split them in the middle). Did I misunderstand, or do you have a
> >solution for that as well.
> >
> Are you sure that it's true? My NTFS resizer interprets parts of runlist 
> stored in different FILE records independently and I never experienced 
> any problems with that.

I'm sure it could happen on NT 3.1. Maybe MS has fixed it since.

Regards,
Martin

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Yuri P. <yu...@ac...>]

Martin von Loewis wrote:

>That would not work: NT would split individual runs across extends
>(i.e. split them in the middle). Did I misunderstand, or do you have a
>solution for that as well.
>
Are you sure that it's true? My NTFS resizer interprets parts of runlist 
stored in different FILE records independently and I never experienced 
any problems with that.


Yuri

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Anton A. <ai...@ca...>]

At 13:53 27/05/2001, Martin von Loewis wrote:
> > Yes and no. They will be uncompressed but not when opening the inode. It
> > will be "uncompress required extent's run list on demand".
>
>Are you sure this can work?

No.

>Initially, I thought I could use the attribute list to only uncompress the 
>extend that has the VCN I'm interested in.

Same idea here.

>That would not work: NT would split individual runs across extends
>(i.e. split them in the middle).

Argh! That seems really stupid thing to do, as it makes it difficult to 
interpret what the highest_vcn/lowest_vcn field of attribute extents is 
supposed to mean!?!

This does however explain some of the code uglyness I have seen (and chosen 
to ignore) in the ntfs.sys disassembly run list handling...

>Did I misunderstand, or do you have a solution for that as well.

No solution. I wasn't aware this could happen. I knew a compression block 
could be split in it halves but I didn't realize this braindamaged 
complication.

I guess we will have to decode the whole run list in one go then. Anything 
else would be slower over all. - What we could do of course is to walk the 
mapping pairs array real quick not reading the numbers only to get the 
correct starting offset into the next extent and then only decode that one 
but that would mean walking the mapping pairs array repeatedly (once to get 
each extent) which would be overall slower than just getting the lot at 
once. - I maintain that we should do this on demand and not on inode open, 
though. - I will have another think about this, but if it is true that NT 
splits the records anywhere, then it would be impossible to start at any 
extent other than the first one.

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://sf.net/projects/linux-ntfs/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Martin v. L. <lo...@in...>]

> Yes and no. They will be uncompressed but not when opening the inode. It 
> will be "uncompress required extent's run list on demand".

Are you sure this can work? Initially, I thought I could use the
attribute list to only uncompress the extend that has the VCN I'm
interested in.

That would not work: NT would split individual runs across extends
(i.e. split them in the middle). Did I misunderstand, or do you have a
solution for that as well.

Regards,
Martin

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Anton A. <ai...@ca...>]

At 12:23 27/05/2001, Martin von Loewis wrote:
> > I would love to move inode metadata to the page cache. That would simplify
> > so much in the NTFS driver and would result in an incredible speed boost
> > due to getting rid of all the silly copying of data back and forth inside
> > the driver as we could just pass pointers around instead (to locked pages
> > or other form of internal locking).
>[...]
> > No thank you. I would rather have it contiguous or at least logically
> > contiguous. I don't care if the pages are physically contiguous as long as
> > I can see them as one piece... I mean, ok, I could break up run lists, as
> > they are arrays of fixed size structs and the boundary cases would be
> > predictable
>
>When you talk about avoiding copying, are you thinking of not copying
>the runlists, as well? If so, how can you avoid uncompressing them?
>That seems very complicated, plus you have to concatenate the pieces
>of the runlist if they span multiple MFT records, anyway.

No, you can't avoid uncompressing them. But you can avoid copying all 
attributes out of the record into separate memory locations and then back. 
The run list is not part of the attribute so obviously has to be kept 
separate. Also, it is pointless to parse all attributes and their run lists 
on opening an inode, chaces are some of the attributes will never be 
touched at all and maybe we are just running stat on the file but don't 
want to start reading/writing the data. My idea involves a different run 
list format (as I already use in the linux-ntfs project on SF) which keeps 
[VCN, LCN, Length] records thus allowing you to not have the whole run 
list, i.e. if user accesses only beginning of the file, we only need to 
decompress the run list for the first extent. Also, this format allows out 
of line attribute extent handling, so if we access data at a certain offset 
we only get that extent and later as required we get the others and insert 
them into the run list. (This is not implemented yet, but I have been 
thinking about it.) Although I have to say I am not sure I want to have the 
run list extents fused into one. That only causes headaches when writing 
back the inode. But perhaps we can fuse them and then discard them without 
write back. Only if the attribute size changes do we need to deal with the 
problem but still it remains a problem.

> > They are not. They are derived from compressed on-disk structures (which
> > are re-compressed when updating the inode). Runlists of such large size 
> are
> > only needed for A) huge files/directories, B) extremely fragmented
> > files/directories, or C) a combination of A and B. (Remembering that
> > metadata is stored as files, so the same applies for the metadata itself.)
>
>I guess this answers my question - you will continue to uncompress the
>runlist when opening the inode, right?

Yes and no. They will be uncompressed but not when opening the inode. It 
will be "uncompress required extent's run list on demand". The current 
driver with uncompressing everything and keeping copies of all attributes 
is A) very slow and B) eats incredible amounts of memory when you are 
operating on large partitions (talking tens of Gb partitions).

Best regards,

Anton


-- 
   "Nothing succeeds like success." - Alexandre Dumas
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://sf.net/projects/linux-ntfs/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: [Linux-ntfs] Re: [Linux-NTFS-Dev] Re: ANN: NTFS new release available (1.1.15)

[Martin v. L. <lo...@in...>]

> I would love to move inode metadata to the page cache. That would simplify 
> so much in the NTFS driver and would result in an incredible speed boost 
> due to getting rid of all the silly copying of data back and forth inside 
> the driver as we could just pass pointers around instead (to locked pages 
> or other form of internal locking).
[...]
> No thank you. I would rather have it contiguous or at least logically 
> contiguous. I don't care if the pages are physically contiguous as long as 
> I can see them as one piece... I mean, ok, I could break up run lists, as 
> they are arrays of fixed size structs and the boundary cases would be 
> predictable

When you talk about avoiding copying, are you thinking of not copying
the runlists, as well? If so, how can you avoid uncompressing them?
That seems very complicated, plus you have to concatenate the pieces
of the runlist if they span multiple MFT records, anyway.

> They are not. They are derived from compressed on-disk structures (which 
> are re-compressed when updating the inode). Runlists of such large size are 
> only needed for A) huge files/directories, B) extremely fragmented 
> files/directories, or C) a combination of A and B. (Remembering that 
> metadata is stored as files, so the same applies for the metadata itself.)

I guess this answers my question - you will continue to uncompress the
runlist when opening the inode, right?

Regards,
Martin