|
From: Anton A. <ai...@ca...> - 2007-09-14 19:02:19
|
Hi, On 14 Sep 2007, at 16:21, mar...@es... wrote: >> On 13 Sep 2007, at 21:35, mar...@es... wrote: >>> When copying large amounts of data (> 25GB) from an NTFS mount, my >>> kernel >>> panics. >> >> Ouch. Would it be possible for you to write down and email the >> actual panic text? (Or capture it via serial console if you don't >> want to type it by hand...) > > I guess I may have stated that wrong, I don't think the ntfs driver is > making the kernel panic directly, I think it is using up all of the > memory > and then when another process tries to use some memory, that invokes > oom-killer. It doesn't seem like it always makes the kernel panic, > but I > think it is init that invokes oom-killer because there is no more > memory: > init invoked oom-killer > > Is that correct? Not necessarily. It may well be ext3 or some other bug in the kernel that is causing it, not necessarily NTFS at all... > Here is the message I get when the kernel panics: > Kernel panic - not syncing: Out of memory and no killable processes. > .. > Kernel panic - not syncing: Out of memory and no killable processes. Right, ok, thanks. >>> escabot log # uname -a >>> Linux escabot 2.6.21-gentoo-r4 #3 SMP PREEMPT Thu Sep 13 14:44:12 >>> EDT 2007 >>> i686 AMD Opteron(tm) Processor 280 AuthenticAMD GNU/Linux >> >> Out of curiosity. Why are you running a 32-bit kernel on such a >> large 64-bit machine?!? Your machine would run a lot quicker with a >> 64-bit kernel (and 64-bit user space)... > > I really don't have a good reason, I probably should be. To be > honest, I > kind of forgot this was an Opteron until you mentioned that. =) (-: >>> I watch top while copying the files and it just sucks up all of the >>> memory. It gets to a point for a while where it seems to copy fine >>> leaving >>> just a few megabytes of memory free. If I copy smaller sets of >>> data, it >>> seems to work fine (but still sucks all the memory it can). This >>> machine >>> has 12GB of memory. >>> >>> I have tried several times and I can pretty much get it to always >>> dump if >>> I copy enough stuff over. The last time I recompiled the kernel >>> with NTFS >>> debug support and enabled the debugging messages. I am attaching a >>> gzip'd >>> file of the kernel messages. >> >> Great, thanks. >> >>> If attaching a file and sending it to the mailing list is improper >>> etiquette, let me know and I can post it on a web page or do >>> something >>> else. >> >> No that is fine! >> >>> I'm not sure if this is a bug or something I'm doing wrong. >>> >>> Thank you for your time. >> >> The kernel panic()ing is always a kernel bug! >> >> The question is what are you copying to? > > I was copying to an ext3 file system. Ok. That may well be causing the problem given the size of your memory and the fact that you are using a 32-bit kernel because ext3 only can use lowmem, i.e. well less than 1GiB of RAM for buffers and that means it might fill up all of lowmem easily that way when you have so much highmem available. NTFS uses highmem for everything so it should not be at fault. >> You said you are copying those large files from NTFS but never said >> what file system is the destination... >> >> To see whether it is NTFS at fault (I don't think it is, I think it >> is either your destination file system or a bug in the VFS/VM not a >> in a file system at all) try to copy to /dev/null, i.e. if you are >> copying using "cp" command then do: >> >> cp /path/to/ntfs/volume/huge26gbfile /dev/null >> >> If you use "dd" then do: >> >> dd if=/path/to/ntfs/volume/huge26gbfile of=/dev/null [insert your >> usual dd options here] >> >> Also out of interest what command are you exactly running to do the >> copying? >> > > Before trying /dev/null just a: cp -R IMAGES /root > >> That will effectively read in the whole file but throw what it reads >> away instead of writing it to another file system. >> >> Does this work or also panic? > > escabot CAD # ls -ltrh > total 49G > -r-------- 1 root root 2.0G Oct 7 2004 RTC26001.GHS > -r-------- 1 root root 2.0G Oct 7 2004 RTC26002.GHS > -r-------- 1 root root 2.0G Oct 7 2004 RTC2604.GHO > -r-------- 1 root root 620M Oct 7 2004 RTC26003.GHS > -r-------- 1 root root 4 Sep 14 09:32 asdf.txt > -r-------- 1 root root 8 Sep 14 09:33 rty.txt > -r-------- 1 root root 42G Sep 14 10:36 bigfile.gho > escabot CAD # cp bigfile.gho /dev/null > >> My expectation is that it will work fine but there is only one way to >> find out and that is for you to try it! (-: > > Thank you for your time and help. I look forward to hearing from > you soon. You forgot to say what happened!!! Did it panic or did it work when you did the cp to /dev/null? Best regards, Anton -- Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @) Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK Linux NTFS maintainer, http://www.linux-ntfs.org/ |
