|
From: <mar...@es...> - 2007-09-14 19:12:22
|
> Hi, > > On 14 Sep 2007, at 16:21, mar...@es... wrote: >>> On 13 Sep 2007, at 21:35, mar...@es... wrote: >>>> When copying large amounts of data (> 25GB) from an NTFS mount, my >>>> kernel >>>> panics. >>> >>> Ouch. Would it be possible for you to write down and email the >>> actual panic text? (Or capture it via serial console if you don't >>> want to type it by hand...) >> >> I guess I may have stated that wrong, I don't think the ntfs driver is >> making the kernel panic directly, I think it is using up all of the >> memory >> and then when another process tries to use some memory, that invokes >> oom-killer. It doesn't seem like it always makes the kernel panic, >> but I >> think it is init that invokes oom-killer because there is no more >> memory: >> init invoked oom-killer >> >> Is that correct? > > Not necessarily. It may well be ext3 or some other bug in the kernel > that is causing it, not necessarily NTFS at all... > >> Here is the message I get when the kernel panics: >> Kernel panic - not syncing: Out of memory and no killable processes. >> .. >> Kernel panic - not syncing: Out of memory and no killable processes. > > Right, ok, thanks. > >>>> escabot log # uname -a >>>> Linux escabot 2.6.21-gentoo-r4 #3 SMP PREEMPT Thu Sep 13 14:44:12 >>>> EDT 2007 >>>> i686 AMD Opteron(tm) Processor 280 AuthenticAMD GNU/Linux >>> >>> Out of curiosity. Why are you running a 32-bit kernel on such a >>> large 64-bit machine?!? Your machine would run a lot quicker with a >>> 64-bit kernel (and 64-bit user space)... >> >> I really don't have a good reason, I probably should be. To be >> honest, I >> kind of forgot this was an Opteron until you mentioned that. =) > > (-: > >>>> I watch top while copying the files and it just sucks up all of the >>>> memory. It gets to a point for a while where it seems to copy fine >>>> leaving >>>> just a few megabytes of memory free. If I copy smaller sets of >>>> data, it >>>> seems to work fine (but still sucks all the memory it can). This >>>> machine >>>> has 12GB of memory. >>>> >>>> I have tried several times and I can pretty much get it to always >>>> dump if >>>> I copy enough stuff over. The last time I recompiled the kernel >>>> with NTFS >>>> debug support and enabled the debugging messages. I am attaching a >>>> gzip'd >>>> file of the kernel messages. >>> >>> Great, thanks. >>> >>>> If attaching a file and sending it to the mailing list is improper >>>> etiquette, let me know and I can post it on a web page or do >>>> something >>>> else. >>> >>> No that is fine! >>> >>>> I'm not sure if this is a bug or something I'm doing wrong. >>>> >>>> Thank you for your time. >>> >>> The kernel panic()ing is always a kernel bug! >>> >>> The question is what are you copying to? >> >> I was copying to an ext3 file system. > > Ok. That may well be causing the problem given the size of your > memory and the fact that you are using a 32-bit kernel because ext3 > only can use lowmem, i.e. well less than 1GiB of RAM for buffers and > that means it might fill up all of lowmem easily that way when you > have so much highmem available. NTFS uses highmem for everything so > it should not be at fault. > >>> You said you are copying those large files from NTFS but never said >>> what file system is the destination... >>> >>> To see whether it is NTFS at fault (I don't think it is, I think it >>> is either your destination file system or a bug in the VFS/VM not a >>> in a file system at all) try to copy to /dev/null, i.e. if you are >>> copying using "cp" command then do: >>> >>> cp /path/to/ntfs/volume/huge26gbfile /dev/null >>> >>> If you use "dd" then do: >>> >>> dd if=/path/to/ntfs/volume/huge26gbfile of=/dev/null [insert your >>> usual dd options here] >>> >>> Also out of interest what command are you exactly running to do the >>> copying? >>> >> >> Before trying /dev/null just a: cp -R IMAGES /root >> >>> That will effectively read in the whole file but throw what it reads >>> away instead of writing it to another file system. >>> >>> Does this work or also panic? >> >> escabot CAD # ls -ltrh >> total 49G >> -r-------- 1 root root 2.0G Oct 7 2004 RTC26001.GHS >> -r-------- 1 root root 2.0G Oct 7 2004 RTC26002.GHS >> -r-------- 1 root root 2.0G Oct 7 2004 RTC2604.GHO >> -r-------- 1 root root 620M Oct 7 2004 RTC26003.GHS >> -r-------- 1 root root 4 Sep 14 09:32 asdf.txt >> -r-------- 1 root root 8 Sep 14 09:33 rty.txt >> -r-------- 1 root root 42G Sep 14 10:36 bigfile.gho >> escabot CAD # cp bigfile.gho /dev/null >> >>> My expectation is that it will work fine but there is only one way to >>> find out and that is for you to try it! (-: >> >> Thank you for your time and help. I look forward to hearing from >> you soon. > > You forgot to say what happened!!! Did it panic or did it work when > you did the cp to /dev/null? > Whoops! Sorry, it still causes the kernel to panic when I copy to /dev/null. --Marc > Best regards, > > Anton > -- > Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @) > Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK > Linux NTFS maintainer, http://www.linux-ntfs.org/ > > |
