linux-ntfs-user — Discussion and help for users of Linux-NTFS.

[Linux-NTFS-User] ntfsresize makes disk unbootable

[<ma...@we...>]

Hello,

I have a serios problem using ntfsresize:

1) I made an image from my partition with linux "dd"
2) There where 2 partitions on that image, the system image was the first one
3) I removed the second partition, the images was bootable in VMWare
4) with parted I removed all parttitions and recreated a larger first partition (same starting sektor), the image was bootable in VMWare
5) I used ntfsresize to resize the ntfs-filesystem:
losetup --offset 32256 /dev/loop0 /vmware/SBS-flat.vmdk
ntfsresize -s 73373155840 --no-action /dev/loop0
ntfsresize -s 73373155840 /dev/loop0
losetup -d /dev/loop0

6) Now the image ist no more bootable, chkdsk tells about serios errors and a "dir c:" in the rescue-mode shows nothing

Do you have an Idea what I did wrong?

Thanks in advance

Holger


_______________________________________________________________________
EINE FÜR ALLE: die kostenlose WEB.DE-Plattform für Freunde und Deine
Homepage mit eigenem Namen. Jetzt starten! http://unddu.de/?kid=kid@mf2

Re: [Linux-NTFS-User] ntfs-3g report was ZFS para linux?

[Gustavo G. P. <gu...@co...>]

El Miércoles, 14 de Mayo de 2008, Gustavo Guillermo Perez escribió:
> El Miércoles, 14 de Mayo de 2008, escribió:
> > Hi Gustavo,
> >
> > > He probado ntfs-3g y la verdad las velocidades de lectura y escritura
> > > son horribles,
> >
> > Could you please explain what you specifically mean in numbers and acts
> > (how you measure performance)?
>
> Let's do the maths again, I'll gonna try the newest version, and plug again
> a hard drive that have ntfs partitions and read/write files from IDE, SATA,
> and USB BRIDGES.
This is my benchmark:
LAPTOP: GATEWAY, sata hard drive with AHCI driver on kernel 2.6.23
lspci: [see below]
ram: 2GB
Filse Size: ~20GB
ntfs-3g version: 1.810
ddrescue /dev/sda2 /dev/null:~ 38MB/s
average write speed: 30MB/s

Same test with Journaled ext3
average write speed: 27MB/s

Same test with XFS
average write speed: 32MB/s

So, my apologizes a fuse driver can write faster as a kernel driver. But the 
very first versions of the driver was giving me the bad impression.

[Entonces, disculpas un driver hecho con fuse puede escribir tan rápido como 
uno del kernel, pero fueron las primeras versiones de ntfs3g las que me 
dieron esa impresión.]

00:00.0 Host bridge: Intel Corporation Mobile Memory Controller Hub (rev 03)
00:02.0 VGA compatible controller: Intel Corporation Mobile Integrated 
Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation Mobile Integrated Graphics 
Controller (rev 03)
00:1a.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI 
Contoller #4 (rev 03)
00:1a.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI 
Controller #5 (rev 03)
00:1a.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI 
Controller #2 (rev 03)
00:1b.0 Audio device: Intel Corporation 82801H (ICH8 Family) HD Audio 
Controller (rev 03)
00:1c.0 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 1 
(rev 03)
00:1c.2 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 3 
(rev 03)
00:1c.5 PCI bridge: Intel Corporation 82801H (ICH8 Family) PCI Express Port 6 
(rev 03)
00:1d.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI 
Controller #1 (rev 03)
00:1d.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI 
Controller #2 (rev 03)
00:1d.2 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI 
Controller #3 (rev 03)
00:1d.7 USB Controller: Intel Corporation 82801H (ICH8 Family) USB2 EHCI 
Controller #1 (rev 03)
00:1e.0 PCI bridge: Intel Corporation 82801 Mobile PCI Bridge (rev f3)
00:1f.0 ISA bridge: Intel Corporation Mobile LPC Interface Controller (rev 03)
00:1f.1 IDE interface: Intel Corporation Mobile IDE Controller (rev 03)
00:1f.2 SATA controller: Intel Corporation Mobile SATA AHCI Controller (rev 
03)
00:1f.3 SMBus: Intel Corporation 82801H (ICH8 Family) SMBus Controller (rev 
03)
02:00.0 Network controller: Intel Corporation PRO/Wireless 3945ABG Network 
Connection (rev 02)
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8101E PCI 
Express Fast Ethernet controller (rev 01)

> Best Regards, by the comment.
>
> > Unfortunately users report just the opposite to us and we can also only
> > reproduce comparable numbers to in-kernel file systems with the still
> > completely unoptimized NTFS-3G driver.
> >
> > If you find horrible performance then we would optimize it. But we can't
> > if nobody reports it to us. Thanks a lot in advance!
> >
> > Some known issues:
> >
> > 	http://ntfs-3g.org/support.html#cpu100
> > 	http://lkml.org/lkml/2008/4/18/136
> >
> > Best regards,
> > 		Szaka
> >
> > --
> > NTFS-3G:  http://ntfs-3g.org



-- 
Gustavo Guillermo Pérez
Compunauta uLinux
www.compunauta.com

Re: [Linux-NTFS-User] ZFS para linux?

[Gustavo G. P. <gu...@co...>]

El Miércoles, 14 de Mayo de 2008, escribió:
> Hi Gustavo,
>
> > He probado ntfs-3g y la verdad las velocidades de lectura y escritura son
> > horribles,
>
> Could you please explain what you specifically mean in numbers and acts
> (how you measure performance)?
Let's do the maths again, I'll gonna try the newest version, and plug again a 
hard drive that have ntfs partitions and read/write files from IDE, SATA, and 
USB BRIDGES.

Best Regards, by the comment.
> Unfortunately users report just the opposite to us and we can also only
> reproduce comparable numbers to in-kernel file systems with the still
> completely unoptimized NTFS-3G driver.
>
> If you find horrible performance then we would optimize it. But we can't if
> nobody reports it to us. Thanks a lot in advance!
>
> Some known issues:
>
> 	http://ntfs-3g.org/support.html#cpu100
> 	http://lkml.org/lkml/2008/4/18/136
>
> Best regards,
> 		Szaka
>
> --
> NTFS-3G:  http://ntfs-3g.org



-- 
Gustavo Guillermo Pérez
Compunauta uLinux
www.compunauta.com

Re: [Linux-NTFS-User] Cloning and resizing with image files

[Anton A. <ai...@ca...>]

Hi,

On 20 Sep 2007, at 20:26, Chris Johns wrote:
> I'm trying to use ntfsclone to copy an NTFS disk partition to an image
> file, then resize the image file to fit a larger (or smaller, within
> reason) partition and then use ntfsclone again to restore the resized
> image file contents to the new partition.
>
> Of note is that I'm using the saved-image format for the image file.
>
> However:
>
> # ntfsclone -o /tmp/ntfs1.img -s /dev/sdd1
> ntfsclone v1.9.4
> NTFS volume version: 3.1
> Cluster size       : 4096 bytes
> Current volume size: 36405055488 bytes (36406 MB)
> Current device size: 36405057024 bytes (36406 MB)
> Scanning volume ...
> 100.00 percent completed
> Accounting clusters ...
> Space in use       : 3020 MB (8.3%)
> Saving NTFS to image ...
> 100.00 percent completed
> Syncing ...
>
> Fine so far. But then if I try to use ntfsresize:
>
> # ntfsresize -n -i /tmp/ntfs1.img
> ntfsresize v1.9.4
> ERROR(22): ntfs_mount failed: Invalid argument
> Apparently device '/tmp/ntfs1.img' doesn't have a valid NTFS.
> Maybe you selected the wrong partition? Or the whole disk instead of a
> partition (e.g. /dev/hda, not /dev/hda1)? This error might also occur
> if the disk was incorrectly repartitioned (see the ntfsresize FAQ).
>
> Does ntfsresize work on 'save-image' files,

No.

> or only on regular clones of the filesystem?

Yes.

> If it doesn't work, is there an alternative that doesn't involve  
> resizing or writing to the original disk partition, and that also  
> doesn't force me to clone a huge filesystem to an equally huge  
> image file?

Not that I know of.

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

[Linux-NTFS-User] Cloning and resizing with image files

[Chris J. <cb...@ca...>]

Hi,

I'm trying to use ntfsclone to copy an NTFS disk partition to an image 
file, then resize the image file to fit a larger (or smaller, within 
reason) partition and then use ntfsclone again to restore the resized 
image file contents to the new partition.

Of note is that I'm using the saved-image format for the image file.

However:

# ntfsclone -o /tmp/ntfs1.img -s /dev/sdd1
ntfsclone v1.9.4
NTFS volume version: 3.1
Cluster size       : 4096 bytes
Current volume size: 36405055488 bytes (36406 MB)
Current device size: 36405057024 bytes (36406 MB)
Scanning volume ...
100.00 percent completed
Accounting clusters ...
Space in use       : 3020 MB (8.3%)  
Saving NTFS to image ...
100.00 percent completed
Syncing ...

Fine so far. But then if I try to use ntfsresize:

# ntfsresize -n -i /tmp/ntfs1.img
ntfsresize v1.9.4
ERROR(22): ntfs_mount failed: Invalid argument
Apparently device '/tmp/ntfs1.img' doesn't have a valid NTFS.
Maybe you selected the wrong partition? Or the whole disk instead of a
partition (e.g. /dev/hda, not /dev/hda1)? This error might also occur
if the disk was incorrectly repartitioned (see the ntfsresize FAQ).

Does ntfsresize work on 'save-image' files, or only on regular clones of 
the filesystem? If it doesn't work, is there an alternative that doesn't 
involve resizing or writing to the original disk partition, and that 
also doesn't force me to clone a huge filesystem to an equally huge 
image file?

Thanks,

Chris

Re: [Linux-NTFS-User] Problem with ntfsdecrypt and PKCS#1 padding

[Anton A. <ai...@ca...>]

And could you either search and replace all occurrences of  
"ntfs_log_debug" with "ntfs_log_info" in ntfsdecrypt.c or apply the  
attached patch on top of the previous one?  This is so we see a bit  
more what is going on...

Thanks,

	Anton

Re: [Linux-NTFS-User] Problem with ntfsdecrypt and PKCS#1 padding

[Anton A. <ai...@ca...>]

Hi Matthew,

On 17 Sep 2007, at 16:10, Matthew A. Postiff wrote:
> Thanks for the quick replies. /dev/sda1 is an external USB hard  
> drive.  The copying involved simply dragging/dropping various  
> unencrypted directories/files from my WinXP documents dir to the  
> external USB drive. Then I right-clicked those directories on the  
> USB drive and told XP to encrypt them.

Great!  I was worried when you said backup that you used some backup  
application that went and did something weird...

> Since my XP system crashed, I unhooked the USB drive from it and  
> attached it as /dev/sda1 to my Linux system.
>
> I now understand what you are saying about the FEK being in the  
> MFT, though I don't know the significance of that fact in my  
> particular case.

The FEK is the key that allows you to decrypt the file.  Without it  
it is impossible to do the decryption (unless you were to brute force  
it and happened to have more computing power than all computers on  
the planet put together at your disposal!).

The FEK is stored in the $LOGGED_UTILITY_STREAM:$EFS attribute in the  
MFT record of each encrypted file.

Things are not quite so simple as the FEK is encrypted with another  
key (this is an RSA key).  Thus you need the RSA private key to be  
able to decrypt the FEK and thus to be able to decrypt the file.

The RSA private key is what is stored in the .pfx file when you  
export your private key in Windows hence why we need the .pfx file.   
We parse it then decrypt the .pfx file using the password you give to  
ntfsdecrypt then this gives us the RSA key which we use to decrypt  
the FEK in the $EFS attribute and then we use the decrypted FEK to  
decrypt the file data.

> I applied your patch, rebuilt, and re-ran ntfsdecrypt, with exactly  
> the same result as in my previous email.

Sorry my patch did not quite do what I intended!  Could you revert  
that patch and apply the attached one instead and try again?  Thanks!

Re: [Linux-NTFS-User] Problem with ntfsdecrypt and PKCS#1 padding

[Matthew A. P. <pos...@co...>]

Hi Anton,

Thanks for the quick replies. /dev/sda1 is an external USB hard drive.
The copying involved simply dragging/dropping various unencrypted
directories/files from my WinXP documents dir to the external USB drive.
Then I right-clicked those directories on the USB drive and told XP to
encrypt them. Since my XP system crashed, I unhooked the USB drive from
it and attached it as /dev/sda1 to my Linux system.

I now understand what you are saying about the FEK being in the MFT,
though I don't know the significance of that fact in my particular case.

I applied your patch, rebuilt, and re-ran ntfsdecrypt, with exactly the
same result as in my previous email.

ntfsinfo /dev/sda1 -vvvF "dir/file"  > ntfsinfo.output.txt
    is attached.

Matt

Anton Altaparmakov wrote:
> Hi Matthew,
>
> On 17 Sep 2007, at 15:20, Matthew A. Postiff wrote:
>> I rebuilt from a fresh cvs checkout just now:
>>
>> % ntfsdecrypt --version
>>
>> ntfsdecrypt v2.0.0 (libntfs 10:0:0) - Decrypt files and print on the
>> standard output.
>>
>> % ./ntfsdecrypt -k /home/...pfx /dev/sda1 "dir/file"
>> Enter the password with which the private key was encrypted: *********
>> There are no entries in the DRF array.
>> Failed to obtain file encryption key.  Aborting.
>>
>> I was sort of afraid this might be the outcome, but I'm not sure what
>> to do at this point. The files on /dev/sda1 were copied from Windows
>> XP as encrypted backup files.
>
> Sorry I do not understand what you mean.  Could you explain what this
> copying involved?
>
>> Later my original drive has crashed, but I had saved off the PFX file
>> and had my password stored safely away. So I don't have access to the
>> original profile on that machine with its SID or whatever. Is it the
>> case I might not have backed up certain key files (i.e. the one
>> containing the FEK)? If so, am I hosed or is there still hope?
>
> The FEK is inside the MFT record of the encrypted file...
>
> Two things to try:
>
> 1) Apply attached patch to you CVS checkout of ntfsdecrypt.c,
> recompile and try again.
>
>
> 2) Run: ntfsinfo /dev/sda1 -vvvF "dir/file"
> And send the output it generates.
>
> Best regards,
>
>     Anton

Re: [Linux-NTFS-User] Problem with ntfsdecrypt and PKCS#1 padding

[Anton A. <ai...@ca...>]

Hi Matthew,

On 17 Sep 2007, at 15:20, Matthew A. Postiff wrote:
> I rebuilt from a fresh cvs checkout just now:
>
> % ntfsdecrypt --version
>
> ntfsdecrypt v2.0.0 (libntfs 10:0:0) - Decrypt files and print on the
> standard output.
>
> % ./ntfsdecrypt -k /home/...pfx /dev/sda1 "dir/file"
> Enter the password with which the private key was encrypted: *********
> There are no entries in the DRF array.
> Failed to obtain file encryption key.  Aborting.
>
> I was sort of afraid this might be the outcome, but I'm not sure  
> what to do at this point. The files on /dev/sda1 were copied from  
> Windows XP as encrypted backup files.

Sorry I do not understand what you mean.  Could you explain what this  
copying involved?

> Later my original drive has crashed, but I had saved off the PFX  
> file and had my password stored safely away. So I don't have access  
> to the original profile on that machine with its SID or whatever.  
> Is it the case I might not have backed up certain key files (i.e.  
> the one containing the FEK)? If so, am I hosed or is there still hope?

The FEK is inside the MFT record of the encrypted file...

Two things to try:

1) Apply attached patch to you CVS checkout of ntfsdecrypt.c,  
recompile and try again.

Re: [Linux-NTFS-User] Problem with ntfsdecrypt and PKCS#1 padding

[Matthew A. P. <pos...@co...>]

Hi Anton,

I rebuilt from a fresh cvs checkout just now:

% ntfsdecrypt --version

ntfsdecrypt v2.0.0 (libntfs 10:0:0) - Decrypt files and print on the
standard output.

% ./ntfsdecrypt -k /home/...pfx /dev/sda1 "dir/file"
Enter the password with which the private key was encrypted: *********
There are no entries in the DRF array.
Failed to obtain file encryption key.  Aborting.

I was sort of afraid this might be the outcome, but I'm not sure what to
do at this point. The files on /dev/sda1 were copied from Windows XP as
encrypted backup files. Later my original drive has crashed, but I had
saved off the PFX file and had my password stored safely away. So I
don't have access to the original profile on that machine with its SID
or whatever. Is it the case I might not have backed up certain key files
(i.e. the one containing the FEK)? If so, am I hosed or is there still hope?

Matt

Anton Altaparmakov wrote:
> Hi,
>
> On 17 Sep 2007, at 14:34, Matthew A. Postiff wrote:
>> % ntfsdecrypt --version
>>
>> ntfsdecrypt v1.13.1 (libntfs 9:0:0) - Decrypt files and print on the
>> standard output.
>>
>> % ntfsdecrypt -k /home/...pfx /dev/sda1 "dir/file"
>> Enter the password with which the private key was encrypted: **********
>> Failed to remove PKCS#1 padding from decrypted file encryption key.
>> Failed to decrypt the file encryption key.
>> Failed to obtain file encryption key.  Aborting.
>>
>> What can I do now?
>
> Please try the latest version (in CVS) or even better wait a few days
> for us to release ntfsprogs 2.0.0 and then try with them.
>
> Best regards,
>
>     Anton

Re: [Linux-NTFS-User] Problem with ntfsdecrypt and PKCS#1 padding

[Anton A. <ai...@ca...>]

Hi,

On 17 Sep 2007, at 14:34, Matthew A. Postiff wrote:
> % ntfsdecrypt --version
>
> ntfsdecrypt v1.13.1 (libntfs 9:0:0) - Decrypt files and print on the
> standard output.
>
> % ntfsdecrypt -k /home/...pfx /dev/sda1 "dir/file"
> Enter the password with which the private key was encrypted:  
> **********
> Failed to remove PKCS#1 padding from decrypted file encryption key.
> Failed to decrypt the file encryption key.
> Failed to obtain file encryption key.  Aborting.
>
> What can I do now?

Please try the latest version (in CVS) or even better wait a few days  
for us to release ntfsprogs 2.0.0 and then try with them.

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

[Linux-NTFS-User] Problem with ntfsdecrypt and PKCS#1 padding

[Matthew A. P. <pos...@co...>]

% ntfsdecrypt --version

ntfsdecrypt v1.13.1 (libntfs 9:0:0) - Decrypt files and print on the
standard output.

% ntfsdecrypt -k /home/...pfx /dev/sda1 "dir/file"
Enter the password with which the private key was encrypted: **********
Failed to remove PKCS#1 padding from decrypted file encryption key.
Failed to decrypt the file encryption key.
Failed to obtain file encryption key.  Aborting.

What can I do now?

Matt

Re: [Linux-NTFS-User] Copying From NTFS Mount & oom-killer

[<mar...@es...>]

> Hi,
>
> On 14 Sep 2007, at 21:27, mar...@es... wrote:
>>> On 14 Sep 2007, at 20:12, mar...@es... wrote:
>>>>>> escabot CAD # ls -ltrh
>>>>>> total 49G
>>>>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26001.GHS
>>>>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26002.GHS
>>>>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC2604.GHO
>>>>>> -r-------- 1 root root 620M Oct  7  2004 RTC26003.GHS
>>>>>> -r-------- 1 root root    4 Sep 14 09:32 asdf.txt
>>>>>> -r-------- 1 root root    8 Sep 14 09:33 rty.txt
>>>>>> -r-------- 1 root root  42G Sep 14 10:36 bigfile.gho
>>>>>> escabot CAD # cp bigfile.gho /dev/null
>>>>
>>>> Whoops! Sorry, it still causes the kernel to panic when I copy to /
>>>> dev/null.
>>>
>>> Hm.  There goes my theory.  Either it must be NTFS itself that is at
>>> fault or it is a bug in the VFS/VM.
>>>
>>> Could you now try the following two things and let me know whether
>>> they work/panic?
>>>
>>> 1) Reboot and place "mem=2G" on the kernel command line (this makes
>>> the kernel use only 2G of your RAM instead of all 12G).  Then try the
>>> copy of bigfile.gho to /dev/null again.  Does it now work or panic
>>> again?
>>
>> It worked setting 'mem=2G' -- it copied all 42GB to /dev/null.
>> Should I
>> try recompiling a kernel with high memory support disabled now, or
>> was the
>> first test sufficient?
>
> Great!  Thanks!  The first test was sufficient.
>
> I think this must be a kernel memory management bug and NTFS just
> happens to trigger it.  Now at least you have a workaround (you can
> try higher values than 2G btw - I have no idea where the magic number
> will be that makes it to run out of memory).  Also, given the nature
> of the bug I would expect that if you upgrade to a 64-bit kernel (you
> will probably need an at least partially 64-bit userspace to match)
> this bug will no longer exist.

Okay, I'll experiment and see what works. Thanks for your help.

>
> If you are happy with that, I will repost your original report with
> your testing results on the kernel mailing list and CC: you so you
> can answer any questions people there might come up with.  Let me
> know if you are happy for me to do this and I will do it tomorrow.
> Or alternatively just send it yourself...  (I am off to bed...)

Yeah, I would appreciate that. Thanks again for all your help Anton.


--Marc

>
> Best regards,
>
> 	Anton
> --
> Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
> Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
> Linux NTFS maintainer, http://www.linux-ntfs.org/
>
>

Re: [Linux-NTFS-User] Copying From NTFS Mount & oom-killer

[Anton A. <ai...@ca...>]

Hi,

On 14 Sep 2007, at 21:27, mar...@es... wrote:
>> On 14 Sep 2007, at 20:12, mar...@es... wrote:
>>>>> escabot CAD # ls -ltrh
>>>>> total 49G
>>>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26001.GHS
>>>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26002.GHS
>>>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC2604.GHO
>>>>> -r-------- 1 root root 620M Oct  7  2004 RTC26003.GHS
>>>>> -r-------- 1 root root    4 Sep 14 09:32 asdf.txt
>>>>> -r-------- 1 root root    8 Sep 14 09:33 rty.txt
>>>>> -r-------- 1 root root  42G Sep 14 10:36 bigfile.gho
>>>>> escabot CAD # cp bigfile.gho /dev/null
>>>
>>> Whoops! Sorry, it still causes the kernel to panic when I copy to /
>>> dev/null.
>>
>> Hm.  There goes my theory.  Either it must be NTFS itself that is at
>> fault or it is a bug in the VFS/VM.
>>
>> Could you now try the following two things and let me know whether
>> they work/panic?
>>
>> 1) Reboot and place "mem=2G" on the kernel command line (this makes
>> the kernel use only 2G of your RAM instead of all 12G).  Then try the
>> copy of bigfile.gho to /dev/null again.  Does it now work or panic
>> again?
>
> It worked setting 'mem=2G' -- it copied all 42GB to /dev/null.  
> Should I
> try recompiling a kernel with high memory support disabled now, or  
> was the
> first test sufficient?

Great!  Thanks!  The first test was sufficient.

I think this must be a kernel memory management bug and NTFS just  
happens to trigger it.  Now at least you have a workaround (you can  
try higher values than 2G btw - I have no idea where the magic number  
will be that makes it to run out of memory).  Also, given the nature  
of the bug I would expect that if you upgrade to a 64-bit kernel (you  
will probably need an at least partially 64-bit userspace to match)  
this bug will no longer exist.

If you are happy with that, I will repost your original report with  
your testing results on the kernel mailing list and CC: you so you  
can answer any questions people there might come up with.  Let me  
know if you are happy for me to do this and I will do it tomorrow.   
Or alternatively just send it yourself...  (I am off to bed...)

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

Re: [Linux-NTFS-User] Copying From NTFS Mount & oom-killer

[<mar...@es...>]

> Hi,
>
> On 14 Sep 2007, at 20:12, mar...@es... wrote:
>>>> escabot CAD # ls -ltrh
>>>> total 49G
>>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26001.GHS
>>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26002.GHS
>>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC2604.GHO
>>>> -r-------- 1 root root 620M Oct  7  2004 RTC26003.GHS
>>>> -r-------- 1 root root    4 Sep 14 09:32 asdf.txt
>>>> -r-------- 1 root root    8 Sep 14 09:33 rty.txt
>>>> -r-------- 1 root root  42G Sep 14 10:36 bigfile.gho
>>>> escabot CAD # cp bigfile.gho /dev/null
>>
>> Whoops! Sorry, it still causes the kernel to panic when I copy to /
>> dev/null.
>
> Hm.  There goes my theory.  Either it must be NTFS itself that is at
> fault or it is a bug in the VFS/VM.
>
> Could you now try the following two things and let me know whether
> they work/panic?
>
> 1) Reboot and place "mem=2G" on the kernel command line (this makes
> the kernel use only 2G of your RAM instead of all 12G).  Then try the
> copy of bigfile.gho to /dev/null again.  Does it now work or panic
> again?
>

It worked setting 'mem=2G' -- it copied all 42GB to /dev/null. Should I
try recompiling a kernel with high memory support disabled now, or was the
first test sufficient?

--Marc

> 2) Given you are compiling your own kernels already, could you please
> try the recompiling the kernel but this time disable high memory
> support completely.  This causes the kernel to only use less than 1G
> of RAM and memory is then used differently.  Then try to copy of
> bigfile.gho to /dev/null again.  Does it now work or panic again?
>
> Thanks a lot for doing all this!
>
> Best regards,
>
> 	Anton
> --
> Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
> Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
> Linux NTFS maintainer, http://www.linux-ntfs.org/
>
>

Re: [Linux-NTFS-User] Copying From NTFS Mount & oom-killer

[Anton A. <ai...@ca...>]

Hi,

On 14 Sep 2007, at 20:12, mar...@es... wrote:
>>> escabot CAD # ls -ltrh
>>> total 49G
>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26001.GHS
>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26002.GHS
>>> -r-------- 1 root root 2.0G Oct  7  2004 RTC2604.GHO
>>> -r-------- 1 root root 620M Oct  7  2004 RTC26003.GHS
>>> -r-------- 1 root root    4 Sep 14 09:32 asdf.txt
>>> -r-------- 1 root root    8 Sep 14 09:33 rty.txt
>>> -r-------- 1 root root  42G Sep 14 10:36 bigfile.gho
>>> escabot CAD # cp bigfile.gho /dev/null
>
> Whoops! Sorry, it still causes the kernel to panic when I copy to / 
> dev/null.

Hm.  There goes my theory.  Either it must be NTFS itself that is at  
fault or it is a bug in the VFS/VM.

Could you now try the following two things and let me know whether  
they work/panic?

1) Reboot and place "mem=2G" on the kernel command line (this makes  
the kernel use only 2G of your RAM instead of all 12G).  Then try the  
copy of bigfile.gho to /dev/null again.  Does it now work or panic  
again?

2) Given you are compiling your own kernels already, could you please  
try the recompiling the kernel but this time disable high memory  
support completely.  This causes the kernel to only use less than 1G  
of RAM and memory is then used differently.  Then try to copy of  
bigfile.gho to /dev/null again.  Does it now work or panic again?

Thanks a lot for doing all this!

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

Re: [Linux-NTFS-User] Copying From NTFS Mount & oom-killer

[<mar...@es...>]

> Hi,
>
> On 14 Sep 2007, at 16:21, mar...@es... wrote:
>>> On 13 Sep 2007, at 21:35, mar...@es... wrote:
>>>> When copying large amounts of data (> 25GB) from an NTFS mount, my
>>>> kernel
>>>> panics.
>>>
>>> Ouch.  Would it be possible for you to write down and email the
>>> actual panic text?  (Or capture it via serial console if you don't
>>> want to type it by hand...)
>>
>> I guess I may have stated that wrong, I don't think the ntfs driver is
>> making the kernel panic directly, I think it is using up all of the
>> memory
>> and then when another process tries to use some memory, that invokes
>> oom-killer. It doesn't seem like it always makes the kernel panic,
>> but I
>> think it is init that invokes oom-killer because there is no more
>> memory:
>> init invoked oom-killer
>>
>> Is that correct?
>
> Not necessarily.  It may well be ext3 or some other bug in the kernel
> that is causing it, not necessarily NTFS at all...
>
>> Here is the message I get when the kernel panics:
>> Kernel panic - not syncing: Out of memory and no killable processes.
>> ..
>> Kernel panic - not syncing: Out of memory and no killable processes.
>
> Right, ok, thanks.
>
>>>> escabot log # uname -a
>>>> Linux escabot 2.6.21-gentoo-r4 #3 SMP PREEMPT Thu Sep 13 14:44:12
>>>> EDT 2007
>>>> i686 AMD Opteron(tm) Processor 280 AuthenticAMD GNU/Linux
>>>
>>> Out of curiosity.  Why are you running a 32-bit kernel on such a
>>> large 64-bit machine?!?  Your machine would run a lot quicker with a
>>> 64-bit kernel (and 64-bit user space)...
>>
>> I really don't have a good reason, I probably should be. To be
>> honest, I
>> kind of forgot this was an Opteron until you mentioned that. =)
>
> (-:
>
>>>> I watch top while copying the files and it just sucks up all of the
>>>> memory. It gets to a point for a while where it seems to copy fine
>>>> leaving
>>>> just a few megabytes of memory free. If I copy smaller sets of
>>>> data, it
>>>> seems to work fine (but still sucks all the memory it can). This
>>>> machine
>>>> has 12GB of memory.
>>>>
>>>> I have tried several times and I can pretty much get it to always
>>>> dump if
>>>> I copy enough stuff over. The last time I recompiled the kernel
>>>> with NTFS
>>>> debug support and enabled the debugging messages. I am attaching a
>>>> gzip'd
>>>> file of the kernel messages.
>>>
>>> Great, thanks.
>>>
>>>> If attaching a file and sending it to the mailing list is improper
>>>> etiquette, let me know and I can post it on a web page or do
>>>> something
>>>> else.
>>>
>>> No that is fine!
>>>
>>>> I'm not sure if this is a bug or something I'm doing wrong.
>>>>
>>>> Thank you for your time.
>>>
>>> The kernel panic()ing is always a kernel bug!
>>>
>>> The question is what are you copying to?
>>
>> I was copying to an ext3 file system.
>
> Ok.  That may well be causing the problem given the size of your
> memory and the fact that you are using a 32-bit kernel because ext3
> only can use lowmem, i.e. well less than 1GiB of RAM for buffers and
> that means it might fill up all of lowmem easily that way when you
> have so much highmem available.  NTFS uses highmem for everything so
> it should not be at fault.
>
>>> You said you are copying those large files from NTFS but never said
>>> what file system is the destination...
>>>
>>> To see whether it is NTFS at fault (I don't think it is, I think it
>>> is either your destination file system or a bug in the VFS/VM not a
>>> in a file system at all) try to copy to /dev/null, i.e. if you are
>>> copying using "cp" command then do:
>>>
>>> cp /path/to/ntfs/volume/huge26gbfile /dev/null
>>>
>>> If you use "dd" then do:
>>>
>>> dd if=/path/to/ntfs/volume/huge26gbfile of=/dev/null [insert your
>>> usual dd options here]
>>>
>>> Also out of interest what command are you exactly running to do the
>>> copying?
>>>
>>
>> Before trying /dev/null just a: cp -R IMAGES /root
>>
>>> That will effectively read in the whole file but throw what it reads
>>> away instead of writing it to another file system.
>>>
>>> Does this work or also panic?
>>
>> escabot CAD # ls -ltrh
>> total 49G
>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26001.GHS
>> -r-------- 1 root root 2.0G Oct  7  2004 RTC26002.GHS
>> -r-------- 1 root root 2.0G Oct  7  2004 RTC2604.GHO
>> -r-------- 1 root root 620M Oct  7  2004 RTC26003.GHS
>> -r-------- 1 root root    4 Sep 14 09:32 asdf.txt
>> -r-------- 1 root root    8 Sep 14 09:33 rty.txt
>> -r-------- 1 root root  42G Sep 14 10:36 bigfile.gho
>> escabot CAD # cp bigfile.gho /dev/null
>>
>>> My expectation is that it will work fine but there is only one way to
>>> find out and that is for you to try it!  (-:
>>
>> Thank you for your time and help. I look forward to hearing from
>> you soon.
>
> You forgot to say what happened!!!  Did it panic or did it work when
> you did  the cp to /dev/null?
>

Whoops! Sorry, it still causes the kernel to panic when I copy to /dev/null.

--Marc


> Best regards,
>
> 	Anton
> --
> Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
> Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
> Linux NTFS maintainer, http://www.linux-ntfs.org/
>
>

Re: [Linux-NTFS-User] Copying From NTFS Mount & oom-killer

[Anton A. <ai...@ca...>]

Hi,

On 14 Sep 2007, at 16:21, mar...@es... wrote:
>> On 13 Sep 2007, at 21:35, mar...@es... wrote:
>>> When copying large amounts of data (> 25GB) from an NTFS mount, my
>>> kernel
>>> panics.
>>
>> Ouch.  Would it be possible for you to write down and email the
>> actual panic text?  (Or capture it via serial console if you don't
>> want to type it by hand...)
>
> I guess I may have stated that wrong, I don't think the ntfs driver is
> making the kernel panic directly, I think it is using up all of the  
> memory
> and then when another process tries to use some memory, that invokes
> oom-killer. It doesn't seem like it always makes the kernel panic,  
> but I
> think it is init that invokes oom-killer because there is no more  
> memory:
> init invoked oom-killer
>
> Is that correct?

Not necessarily.  It may well be ext3 or some other bug in the kernel  
that is causing it, not necessarily NTFS at all...

> Here is the message I get when the kernel panics:
> Kernel panic - not syncing: Out of memory and no killable processes.
> ..
> Kernel panic - not syncing: Out of memory and no killable processes.

Right, ok, thanks.

>>> escabot log # uname -a
>>> Linux escabot 2.6.21-gentoo-r4 #3 SMP PREEMPT Thu Sep 13 14:44:12
>>> EDT 2007
>>> i686 AMD Opteron(tm) Processor 280 AuthenticAMD GNU/Linux
>>
>> Out of curiosity.  Why are you running a 32-bit kernel on such a
>> large 64-bit machine?!?  Your machine would run a lot quicker with a
>> 64-bit kernel (and 64-bit user space)...
>
> I really don't have a good reason, I probably should be. To be  
> honest, I
> kind of forgot this was an Opteron until you mentioned that. =)

(-:

>>> I watch top while copying the files and it just sucks up all of the
>>> memory. It gets to a point for a while where it seems to copy fine
>>> leaving
>>> just a few megabytes of memory free. If I copy smaller sets of
>>> data, it
>>> seems to work fine (but still sucks all the memory it can). This
>>> machine
>>> has 12GB of memory.
>>>
>>> I have tried several times and I can pretty much get it to always
>>> dump if
>>> I copy enough stuff over. The last time I recompiled the kernel
>>> with NTFS
>>> debug support and enabled the debugging messages. I am attaching a
>>> gzip'd
>>> file of the kernel messages.
>>
>> Great, thanks.
>>
>>> If attaching a file and sending it to the mailing list is improper
>>> etiquette, let me know and I can post it on a web page or do  
>>> something
>>> else.
>>
>> No that is fine!
>>
>>> I'm not sure if this is a bug or something I'm doing wrong.
>>>
>>> Thank you for your time.
>>
>> The kernel panic()ing is always a kernel bug!
>>
>> The question is what are you copying to?
>
> I was copying to an ext3 file system.

Ok.  That may well be causing the problem given the size of your  
memory and the fact that you are using a 32-bit kernel because ext3  
only can use lowmem, i.e. well less than 1GiB of RAM for buffers and  
that means it might fill up all of lowmem easily that way when you  
have so much highmem available.  NTFS uses highmem for everything so  
it should not be at fault.

>> You said you are copying those large files from NTFS but never said
>> what file system is the destination...
>>
>> To see whether it is NTFS at fault (I don't think it is, I think it
>> is either your destination file system or a bug in the VFS/VM not a
>> in a file system at all) try to copy to /dev/null, i.e. if you are
>> copying using "cp" command then do:
>>
>> cp /path/to/ntfs/volume/huge26gbfile /dev/null
>>
>> If you use "dd" then do:
>>
>> dd if=/path/to/ntfs/volume/huge26gbfile of=/dev/null [insert your
>> usual dd options here]
>>
>> Also out of interest what command are you exactly running to do the
>> copying?
>>
>
> Before trying /dev/null just a: cp -R IMAGES /root
>
>> That will effectively read in the whole file but throw what it reads
>> away instead of writing it to another file system.
>>
>> Does this work or also panic?
>
> escabot CAD # ls -ltrh
> total 49G
> -r-------- 1 root root 2.0G Oct  7  2004 RTC26001.GHS
> -r-------- 1 root root 2.0G Oct  7  2004 RTC26002.GHS
> -r-------- 1 root root 2.0G Oct  7  2004 RTC2604.GHO
> -r-------- 1 root root 620M Oct  7  2004 RTC26003.GHS
> -r-------- 1 root root    4 Sep 14 09:32 asdf.txt
> -r-------- 1 root root    8 Sep 14 09:33 rty.txt
> -r-------- 1 root root  42G Sep 14 10:36 bigfile.gho
> escabot CAD # cp bigfile.gho /dev/null
>
>> My expectation is that it will work fine but there is only one way to
>> find out and that is for you to try it!  (-:
>
> Thank you for your time and help. I look forward to hearing from  
> you soon.

You forgot to say what happened!!!  Did it panic or did it work when  
you did  the cp to /dev/null?

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

Re: [Linux-NTFS-User] Copying From NTFS Mount & oom-killer

[<mar...@es...>]

> Hi,
>
> On 13 Sep 2007, at 21:35, mar...@es... wrote:
>> When copying large amounts of data (> 25GB) from an NTFS mount, my
>> kernel
>> panics.
>
> Ouch.  Would it be possible for you to write down and email the
> actual panic text?  (Or capture it via serial console if you don't
> want to type it by hand...)

I guess I may have stated that wrong, I don't think the ntfs driver is
making the kernel panic directly, I think it is using up all of the memory
and then when another process tries to use some memory, that invokes
oom-killer. It doesn't seem like it always makes the kernel panic, but I
think it is init that invokes oom-killer because there is no more memory:
init invoked oom-killer

Is that correct?

Here is the message I get when the kernel panics:
Kernel panic - not syncing: Out of memory and no killable processes.
..
Kernel panic - not syncing: Out of memory and no killable processes.

>
>> escabot log # uname -a
>> Linux escabot 2.6.21-gentoo-r4 #3 SMP PREEMPT Thu Sep 13 14:44:12
>> EDT 2007
>> i686 AMD Opteron(tm) Processor 280 AuthenticAMD GNU/Linux
>
> Out of curiosity.  Why are you running a 32-bit kernel on such a
> large 64-bit machine?!?  Your machine would run a lot quicker with a
> 64-bit kernel (and 64-bit user space)...
>

I really don't have a good reason, I probably should be. To be honest, I
kind of forgot this was an Opteron until you mentioned that. =)

>> I watch top while copying the files and it just sucks up all of the
>> memory. It gets to a point for a while where it seems to copy fine
>> leaving
>> just a few megabytes of memory free. If I copy smaller sets of
>> data, it
>> seems to work fine (but still sucks all the memory it can). This
>> machine
>> has 12GB of memory.
>>
>> I have tried several times and I can pretty much get it to always
>> dump if
>> I copy enough stuff over. The last time I recompiled the kernel
>> with NTFS
>> debug support and enabled the debugging messages. I am attaching a
>> gzip'd
>> file of the kernel messages.
>
> Great, thanks.
>
>> If attaching a file and sending it to the mailing list is improper
>> etiquette, let me know and I can post it on a web page or do something
>> else.
>
> No that is fine!
>
>> I'm not sure if this is a bug or something I'm doing wrong.
>>
>> Thank you for your time.
>
> The kernel panic()ing is always a kernel bug!
>
> The question is what are you copying to?
>

I was copying to an ext3 file system.

> You said you are copying those large files from NTFS but never said
> what file system is the destination...
>
> To see whether it is NTFS at fault (I don't think it is, I think it
> is either your destination file system or a bug in the VFS/VM not a
> in a file system at all) try to copy to /dev/null, i.e. if you are
> copying using "cp" command then do:
>
> cp /path/to/ntfs/volume/huge26gbfile /dev/null
>
> If you use "dd" then do:
>
> dd if=/path/to/ntfs/volume/huge26gbfile of=/dev/null [insert your
> usual dd options here]
>
> Also out of interest what command are you exactly running to do the
> copying?
>

Before trying /dev/null just a: cp -R IMAGES /root

> That will effectively read in the whole file but throw what it reads
> away instead of writing it to another file system.
>
> Does this work or also panic?
>

escabot CAD # ls -ltrh
total 49G
-r-------- 1 root root 2.0G Oct  7  2004 RTC26001.GHS
-r-------- 1 root root 2.0G Oct  7  2004 RTC26002.GHS
-r-------- 1 root root 2.0G Oct  7  2004 RTC2604.GHO
-r-------- 1 root root 620M Oct  7  2004 RTC26003.GHS
-r-------- 1 root root    4 Sep 14 09:32 asdf.txt
-r-------- 1 root root    8 Sep 14 09:33 rty.txt
-r-------- 1 root root  42G Sep 14 10:36 bigfile.gho
escabot CAD # cp bigfile.gho /dev/null


> My expectation is that it will work fine but there is only one way to
> find out and that is for you to try it!  (-:
>
> Best regards,
>
> 	Anton
> --
> Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
> Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
> Linux NTFS maintainer, http://www.linux-ntfs.org/
>

Thank you for your time and help. I look forward to hearing from you soon.

--Marc

>

Re: [Linux-NTFS-User] Copying From NTFS Mount & oom-killer

[Anton A. <ai...@ca...>]

Hi,

On 13 Sep 2007, at 21:35, mar...@es... wrote:
> When copying large amounts of data (> 25GB) from an NTFS mount, my  
> kernel
> panics.

Ouch.  Would it be possible for you to write down and email the  
actual panic text?  (Or capture it via serial console if you don't  
want to type it by hand...)

> escabot log # uname -a
> Linux escabot 2.6.21-gentoo-r4 #3 SMP PREEMPT Thu Sep 13 14:44:12  
> EDT 2007
> i686 AMD Opteron(tm) Processor 280 AuthenticAMD GNU/Linux

Out of curiosity.  Why are you running a 32-bit kernel on such a  
large 64-bit machine?!?  Your machine would run a lot quicker with a  
64-bit kernel (and 64-bit user space)...

> I watch top while copying the files and it just sucks up all of the
> memory. It gets to a point for a while where it seems to copy fine  
> leaving
> just a few megabytes of memory free. If I copy smaller sets of  
> data, it
> seems to work fine (but still sucks all the memory it can). This  
> machine
> has 12GB of memory.
>
> I have tried several times and I can pretty much get it to always  
> dump if
> I copy enough stuff over. The last time I recompiled the kernel  
> with NTFS
> debug support and enabled the debugging messages. I am attaching a  
> gzip'd
> file of the kernel messages.

Great, thanks.

> If attaching a file and sending it to the mailing list is improper
> etiquette, let me know and I can post it on a web page or do something
> else.

No that is fine!

> I'm not sure if this is a bug or something I'm doing wrong.
>
> Thank you for your time.

The kernel panic()ing is always a kernel bug!

The question is what are you copying to?

You said you are copying those large files from NTFS but never said  
what file system is the destination...

To see whether it is NTFS at fault (I don't think it is, I think it  
is either your destination file system or a bug in the VFS/VM not a  
in a file system at all) try to copy to /dev/null, i.e. if you are  
copying using "cp" command then do:

cp /path/to/ntfs/volume/huge26gbfile /dev/null

If you use "dd" then do:

dd if=/path/to/ntfs/volume/huge26gbfile of=/dev/null [insert your  
usual dd options here]

Also out of interest what command are you exactly running to do the  
copying?

That will effectively read in the whole file but throw what it reads  
away instead of writing it to another file system.

Does this work or also panic?

My expectation is that it will work fine but there is only one way to  
find out and that is for you to try it!  (-:

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

[Linux-NTFS-User] Copying From NTFS Mount & oom-killer

[<mar...@es...>]

Hi,

When copying large amounts of data (> 25GB) from an NTFS mount, my kernel
panics.

escabot log # uname -a
Linux escabot 2.6.21-gentoo-r4 #3 SMP PREEMPT Thu Sep 13 14:44:12 EDT 2007
i686 AMD Opteron(tm) Processor 280 AuthenticAMD GNU/Linux

I watch top while copying the files and it just sucks up all of the
memory. It gets to a point for a while where it seems to copy fine leaving
just a few megabytes of memory free. If I copy smaller sets of data, it
seems to work fine (but still sucks all the memory it can). This machine
has 12GB of memory.

I have tried several times and I can pretty much get it to always dump if
I copy enough stuff over. The last time I recompiled the kernel with NTFS
debug support and enabled the debugging messages. I am attaching a gzip'd
file of the kernel messages.

If attaching a file and sending it to the mailing list is improper
etiquette, let me know and I can post it on a web page or do something
else.

I'm not sure if this is a bug or something I'm doing wrong.

Thank you for your time.


--Marc

[Linux-NTFS-User] Need help for kernel upgradation

[Mayuresh M M. <mur...@gm...>]

Hi,

I'm a new bie to kernel,

I want to update my current kernel, 

Reason: i'm not finding the kernel.2.4.20-6.src.i686.rpm ( source file )

I'm trying to install the Ethernet drivers, broadcomm 440x series
I get error saying kernel source not found.

So either i need new kernel with its source, 
Or
Source files for my 2.4.20-6.i686 kernel.

Thanking you in advance...

 
Mayuresh M Murkunde
Gujarat, India
M: +91 - 9898364640
Email: mur...@gm...

[Linux-NTFS-User] mkntfs --partition-start not working?

[Russell S. <rus...@li...>]

Hi everyone,

I'm trying to use mkntfs to format a RAID device /dev/md0.  I am trying to get mkntfs to skip the first sector of /dev/md0 so that I can put a partition table there, but mkntfs is always starting at sector 0 regardless of what I put for the --partition-start value.  I've tried passing in values from 1 to 10 and it always starts at sector 0.

All I want it to do is skip one sector and then lay down the NTFS info.

Has anyone else experienced this problem??

I zero'd out the first few sectors of my disk array:
-----------------------------------------------------------
root@10:/tools> dd if=/dev/zero of=/dev/md0 bs=512 count=100
100+0 records in
100+0 records out
51200 bytes transferred in 0.011349 seconds (4511378 bytes/sec)

Read it back to verify
-----------------------------------------------------------
root@10:/tools> dd if=/dev/md0 of=blk1.blk bs=512 count=1; od blk1.blk
1+0 records in
1+0 records out
512 bytes transferred in 0.018710 seconds (27365 bytes/sec)
0000000 000000 000000 000000 000000 000000 000000 000000 000000
*
0001000
root@10:/tools>

Here's a run of the command with the --debug option turned on:
-----------------------------------------------------------
root@10:/tools>  mkntfs --partition-start 10 --sector-size 512 --fast --debug /dev/md0
sector size = 512 bytes
Used BLKBSZSET to set block size to 512 bytes.
BLKGETSIZE64 nr bytes = 288462929920 (0x4329bb0000)
number of sectors = 563404160 (0x2194dd80)
HDIO_GETGEO sectors_per_track = 4 (0x4)
HDIO_GETGEO heads = 2 (0x2)
volume size = 281702079kiB
Cluster size has been automatically set to 4096 bytes.
cluster size = 4096 bytes
number of clusters = 70425519 (0x4329baf)
System page size is 4096 bytes.
mft record size = 1024 bytes
index record size = 4096 bytes
g_lcn_bitmap_byte_size = 8803192, allocated = 8806400
MFT size = 27648 (0x6c00) bytes
mft_bitmap_size = 27, g_mft_bitmap_byte_size = 8
$MFT logical cluster number = 0x4
MFT zone size = 35212756kiB
$MFTMirr logical cluster number = 0x2194dd7
$LogFile logical cluster number = 0x2194dd8
$LogFile (journal) size = 65536kiB
Creating NTFS volume structures.
Creating root directory (mft record 5)
ntfs_resident_attr_value_resize(): Entering for new size 56.
ntfs_attr_record_resize(): Entering for new_size 88.
Creating $MFT (mft record 0)
Creating $MFTMirr (mft record 1)
Creating $LogFile (mft record 2)
Creating $AttrDef (mft record 4)
Creating $Bitmap (mft record 6)
Creating $Boot (mft record 7)
sectors per track = 4 (0x4)
heads = 2 (0x2)
hidden sectors = 10 (0xa)
clusters per mft record = -10 (0xfffffff6)
clusters per index block = 1 (0x1)

Beginning bootsector check...
Calculating bootsector checksum... OK
Checking OEMid... OK
Checking bytes per sector... OK
Checking sectors per cluster... OK
Checking cluster size... OK
Checking reserved fields are zero... OK
Checking clusters per mft record... OK
Checking clusters per index block... OK
Bootsector check completed successfully.
Creating backup boot sector.
Creating $Volume (mft record 3)
Creating $BadClus (mft record 8)
Creating $Secure (mft record 9)
Creating $UpCase (mft record 0xa)
Creating $Extend (mft record 11)
Creating system file (mft record 0xc)
Creating system file (mft record 0xd)
Creating system file (mft record 0xe)
Creating system file (mft record 0xf)
Creating $Quota (mft record 24)
Creating $ObjId (mft record 25)
Creating $Reparse (mft record 26)
Syncing root directory index record.
ntfs_mapping_pairs_decompress(): Entering for attr 0xa0.
Mapping pairs array successfully decompressed:
NTFS-fs DEBUG: Dumping runlist (values in hex):
VCN              LCN               Run length
0                86537b            1
1                LCN_ENOENT        0                (runlist end)
Syncing $Bitmap.
ntfs_mapping_pairs_decompress(): Entering for attr 0x80.
Mapping pairs array successfully decompressed:
NTFS-fs DEBUG: Dumping runlist (values in hex):
VCN              LCN               Run length
0                86537d            866
866              LCN_ENOENT        0                (runlist end)
Syncing $MFT.
ntfs_pwrite(): Entering for pos 0x4000, count 0x400.
ntfs_pwrite(): Entering for pos 0x4400, count 0x400.
ntfs_pwrite(): Entering for pos 0x4800, count 0x400.
ntfs_pwrite(): Entering for pos 0x4c00, count 0x400.
ntfs_pwrite(): Entering for pos 0x5000, count 0x400.
ntfs_pwrite(): Entering for pos 0x5400, count 0x400.
ntfs_pwrite(): Entering for pos 0x5800, count 0x400.
ntfs_pwrite(): Entering for pos 0x5c00, count 0x400.
ntfs_pwrite(): Entering for pos 0x6000, count 0x400.
ntfs_pwrite(): Entering for pos 0x6400, count 0x400.
ntfs_pwrite(): Entering for pos 0x6800, count 0x400.
ntfs_pwrite(): Entering for pos 0x6c00, count 0x400.
ntfs_pwrite(): Entering for pos 0x7000, count 0x400.
ntfs_pwrite(): Entering for pos 0x7400, count 0x400.
ntfs_pwrite(): Entering for pos 0x7800, count 0x400.
ntfs_pwrite(): Entering for pos 0x7c00, count 0x400.
ntfs_pwrite(): Entering for pos 0x8000, count 0x400.
ntfs_pwrite(): Entering for pos 0x8400, count 0x400.
ntfs_pwrite(): Entering for pos 0x8800, count 0x400.
ntfs_pwrite(): Entering for pos 0x8c00, count 0x400.
ntfs_pwrite(): Entering for pos 0x9000, count 0x400.
ntfs_pwrite(): Entering for pos 0x9400, count 0x400.
ntfs_pwrite(): Entering for pos 0x9800, count 0x400.
ntfs_pwrite(): Entering for pos 0x9c00, count 0x400.
ntfs_pwrite(): Entering for pos 0xa000, count 0x400.
ntfs_pwrite(): Entering for pos 0xa400, count 0x400.
ntfs_pwrite(): Entering for pos 0xa800, count 0x400.
Updating $MFTMirr.
ntfs_pwrite(): Entering for pos 0x2194dd7000, count 0x400.
ntfs_pwrite(): Entering for pos 0x2194dd7400, count 0x400.
ntfs_pwrite(): Entering for pos 0x2194dd7800, count 0x400.
ntfs_pwrite(): Entering for pos 0x2194dd7c00, count 0x400.
Syncing device.
mkntfs completed successfully. Have a nice day.


The 'Hidden Sectors' value is set correctly.  I think that's the number of sectors to skip.

hidden sectors = 10 (0xa)

Now, here's what my sector 0 looks like:
-----------------------------------------------------------
root@10:/tools> dd if=/dev/md0 of=blk1.blk bs=512 count=1; od blk1.blk
1+0 records in
1+0 records out
512 bytes transferred in 0.015100 seconds (33907 bytes/sec)
0000000 055753 047220 043124 020123 020040 000040 004002 000000
0000020 000000 000000 174000 000000 000004 000002 000012 000000
0000040 000000 000000 000200 000200 156577 020624 000000 000000
0000060 000004 000000 000000 000000 046727 001031 000000 000000
0000100 000366 000000 000001 000000 137773 001463 031153 006212
0000120 000000 000000 000000 000000 000000 000000 175000 140063
0000140 150216 000274 175574 140270 107007 143730 052006 000000
0000160 143400 053006 000000 143400 055406 010000 134000 006400
0000200 140216 155453 003750 064000 006400 063150 145402 051520
0000220 051121 063006 052241 063000 003003 000034 031546 063322
0000240 133417 014016 063000 170767 141376 013210 000132 105546
0000260 063320 165301 173420 015066 104000 022426 121400 000130
0000300 014241 025000 055006 040000 003073 000133 001566 055641
0000320 050000 001264 013213 000130 003261 163322 033012 000132
0000340 145213 164606 033212 000045 100262 011715 071130 000452
0000360 052006 101400 053026 000000 003051 000133 005566 160301
0000400 106005 001702 107320 165702 003612 054532 054133 137303
0000420 000531 004353 161676 165401 137003 000471 004750 137000
0000440 000655 001750 175400 177353 036254 072000 132011 135416
0000460 000007 010315 171353 016703 040400 062040 071551 020153
0000500 062562 062141 062440 071162 071157 067440 061543 071165
0000520 062562 027144 005015 024400 040400 065440 071145 062556
0000540 020154 064546 062554 064440 020163 064555 071563 067151
0000560 020147 071146 066557 072040 062550 062040 071551 027153
0000600 005015 022400 040400 065440 071145 062556 020154 064546
0000620 062554 064440 020163 067564 020157 064544 061563 067157
0000640 064564 072547 072557 027163 005015 031400 044400 071556
0000660 071145 020164 020141 074563 072163 066545 062040 071551
0000700 062553 072164 020145 067141 020144 062562 072163 071141
0000720 006564 072012 062550 071440 071571 062564 027155 005015
0000740 013400 056000 052116 042114 020122 071551 061440 066557
0000760 071160 071545 062563 027144 005015 000000 000000 125125
0001000


This should be filled with ZEROS still, right?  This data should be located at sector 10, right?  Everything before sector 10 should have been left alone, right?  

Am I doing something wrong or is this a bug in the tool???  

Thanks!
-Russell

Re: [Linux-NTFS-User] Running ntfsresize non-interactively

[Yura P. <pak...@gm...>]

On Fri, 22 Jun 2007 at 07:56 +0530, K T Ligesh wrote:
> > We try to do not change ntfsresize output without REALLY good reason.
> > BTW, ntfsresize source have following warning in the begging:
> > 
> >  * WARNING FOR DEVELOPERS!!! Several external tools grep for text messages
> >  * to control execution thus if you would like to change any message
> >  * then PLEASE think twice before doing so then don't modify it. Thanks!
> > 
> 
>  I think you should have an option like
> "--i-know-what-the-heck-i-am-doing", which will not ask for
> confirmation. My attempts to use expect has been futile. Can you tell
> me the app that uses ntfsresize in the non-interactive fashion?

AFAIR gparted, qparted and disk partitioning tools in distro installers.

-- 
Best regards,
        Yura

Re: [Linux-NTFS-User] linux ntfs, fedora and xen.

[Yura P. <pak...@gm...>]

On Thu, 21 Jun 2007 at 18:46 +0530, K T Ligesh wrote:
> I hope the devs are aware that ntfs is now integrated with fedora, and
> also that it has become a very important because of Xen/kvm's ability
> to create windows virtual machines, and linux-ntfs becomes an absolute
> must if you need to productively use this facility.  I have read
> somewhere that development activity for linux-ntfs had slowed down.

Yes, there is exist some slowdown. But things really not so bad, just
Anton and I working in cathedral style for now. I will release my work
rather soon (however this no ETA) and Anton (AFAIR) in Spring 2008.

> Is there any chance of it picking up considering the new developments
> with Xen?

I do not see why Xen developers can be interested in NTFS. This two
different projects with no code that can be shared.

-- 
Best regards,
        Yura