Menu

A couple of observations

2008-03-04
2012-09-10
  • JimBob McGee

    JimBob McGee - 2008-03-04

    I've been using G4L to image my laptop since v0.21 and have always found it to be very useful and robust. I have however noticed a couple of things with v0.23 and now v0.24 that I have wondered about.

    (1.) I typically use a RAW/GZip/FTP setup in a 100BaseT LAN but have found, since v0.23, that this combination is very slow. With bzImageb I was getting (on average) 8-10MB/s transfer. With bz22.2 and later I get 1.5MB/s maximum.

    (2.) The above prompted me to try LZOP compression for the first time, where I got 8-10MB/s again but, when I tried to restore, I was told that my image file was not a LZOP file.

    I am sticking with bzImageb -- at least until someone can explain to me the differences; is there anything different between the bzImageb and bz22.x builds that may account for this?

    For reference, I'm backing up a Dell XPS M1210 to an IIS FTP site on a custom-built Windows XP Pro PC:

    • Dell is: 2GHz Core2 Duo, 2GB RAM, 100GB SATA HDD
    • PC is 3GHz P4, 2GB RAM, 250GB SATA HDD

    J.

     
    • Michael Setzer II

      First, glad to here you got something to work.
      Second, it basically is just pipping when you get down to the final process.

      dd if=/dev/xxx | compress program | ncftpput

      and

      ncftpget | compression program | dd of=/dev/xxx

      There is a little more to it, but that is the simplest form

      The only that comes to mind is that it is IIS. One thing that just crossed my mind, is that perhaps it isn't seeing lzo files as being binary, and is doing an ASCII mode and thus losing all the NULL characters from the file. Years ago, I had uploaded a file to a windows machine and it was the same size. After some research, I found that all the NULL characters had been dropped. Only had to add the binary option before the upload.

      Again, I could get IIS to work, and didn't bother to play with it at all. Installed Filezilla on a machine, and it worked first tiem. Also, never had a problem with vsftp on linux even running on an old K62 machine.

      P.S. G4l can not rezie images, but other programs can easily do that. The NTFSCLONE option thou, does allow one to restore an NTFSCLONE backup to a larger partition, and then use ntfsresize to expend to the size of the partition. Had a teache with a full 40GB drive. Did an ntfsclone of it, and restore to a 200GB partition on another drive. Still only 40GB, but then ntfsresize, and it was now using all 200GB.

       
    • Michael Setzer II

      Strange?
      First, I was unaware that anyone had it working with IIS FTP at all. I've had a number of people try it, and have it fail, but after switching to using Filezilla or other ftp server have no problem. Additionally, I've seen the similar problem listed with G4U and IIS FTP.

      Other questions that come to mind, what is the size of the created image file?
      What is the file system on the partition where the iamge is being stored?
      With Fat32, there is a limit of 2GB or 4GB.

      If you still have the lzo compressed file on the system, what does lzop -t imagefile report.

      The only time I've had any problem with a corupted image file, was when I had a system that had bad ram.
      Brand new machine. Worked just fine when running the OS, but would corrupt image files after about 2GB of space. Did the image on another machine, and then compared the created files. Running memtest resulted in 7 of the 8 tests passing with no errors. Then the 8th memory test caused the exact same pattern of failure that I noted in the image. Removed the bad modual and it worked just fine. Vendor the replaced both moduals, and it worked fine.

      My first though is the IIS FTP.
      Second, concerning the kernel, all the kernels on the disk would use the exact same filesystem and directories from the ramdrive.gz, so it would have to be something in the kernel build itself that is causing the difference, but it is build the the kernel.org source, and usually just adds new drivers for the nic and disk controllers.
      Have you tried the bz24.2 or the bz25 git kernel?

      When I image my sytems with lzop compression, with a 100M ethernet, I get speeds of about 30MB/sec, but the drive has generally about 1/2 the disk being cleared sectors, so the compression kicks in.

      Also, if the patition you are backing up is NTFS have you tried NTFSCLONE. I generally do full images of the drive, but also do an NTFSCLONE of the XP partition, that way I can restore my lab systems XP drive in about 10 minutes if it get corrupted by the students. It doesn't include the whole drive, but does make quick work, since it only restores the used data.

      Probable is not an issue, but what version of the ROM are you using in the system? I've seen some messages on the syslinux list about a Dell A11 version of BIOS causing problems, and going back to A10 fixes the problem, but it appears the systems is booting ok?

      The very last thing, is what mode is the disk controller in? I had one system that would not work with later version of g4l, and in checking the bios, it had some special enhanced mode for windows and the disk in the bios. I changed this standard, and g4l then worked just find. I then ran a couple of windows benchmark programs with the setting in both modes, and could find no difference in performance. Could have been the cheap drives that come with the system couldn't handle the enchanced mode anyway?

      Again, more questions than answers, but would be interested if any of this helps to find a solution, but again, my first ideal would be to either confirm or disprove that it is the IIS ftp that is causing the problem for whatever reason. I could never get it to work on my Dell PowerEdge 2500, but filezilla worked fine on the same machine.

      Thanks for taking the time to do testing, and hopefully a solution can be found.

       
      • JimBob McGee

        JimBob McGee - 2008-03-07

        A little more on the LZOP issue I have. I tried an NTFSCLONE/LZOP with 24.3 to my XP/IIS FTP site and got the 'Not a LZOP file' message again when I tried to restore. I tried running the LZOP tool on the file (LZOP for Windows found at http://gnuwin32.sourceforge.net/packages/lzop.htm) and got:

        C:\Temp>lzop -t 20080305.C$.NtfsClone.g4l.lzo
        testing 20080305.c$.ntfsclone.g4l.lzo OK

        C:\Temp>lzop -l 20080305.C$.NtfsClone.g4l.lzo
        method compressed uncompr. ratio uncompressed_name
        LZO1X-1 2979560714 1389582932 214.4% 20080305.c$.ntfsclone.g4l

        and I was also able to uncompress the file. If I try to do an NTFSCLONE restore on the uncompressed file, NTFSCLONE just hangs at the '(libntfs 9:0:0)' line...

        J.

         
    • JimBob McGee

      JimBob McGee - 2008-03-04

      That's interesting, I've been using IIS FTP for some time now and not had a problem. That said, I have always used IIS on Windows Server 2003 R2 -- this is the first time I've tried with IIS on XP. My Windows 2003 FTP is just a regular IIS FTP Site, configured with anonymous access, with read/write permissions.

      To try to answer some of your questions, both source and target filesystems were NTFS, the LZO file was 81GB (it's a near-full drive) and I gave up waiting for the LZOP -t command to finish as I really wanted to do the backup, so deleted the LZO to make space.

      You may be right about the enhanced SATA mode -- it's called AHCI or something -- but I can find no way to check if it is on or turn it off in my BIOS; the XPS BIOS is different from the D620 BIOS that I am used to.

      NTFSCLONE is becoming my new favourite way to do imaging, especially on my Windows 2003 box, which has RAID enabled. The only thing lacking there is an option to turn off the original-partition size-checking (if I have an NTFSCLONE image that is 4GB big, which came from a 145GB partition and I try to restore to a 32GB partition it won't let me). I expect that is a change-request for the NTFSCLONE team rather than yourself, though, but if the option is already there (e.g. a --force option), it would be nice to be able to specify it in G4L (with a check box?!).

      In this case though, I'm trying to get a blanket image of the disk, because I want the full partition layout (including a FAT16 utility partition) so I can mirror it to a much bigger disk (I'll use GParted after restoring to clean up).

      Will get back to you regarding the BIOS ROM -- off the top of my head, it is A05 or A06, but I'm probably wrong.

      Also, as a side issue, I have experienced a number of FTP write failures/client resets with the 0.24 release that I did not get with the 0.21 release (even in bzImageb), so this might be kernel-related too (forgive me for not being able to diagnose that, I'm far from fluent in Linux). It may just be this god-awful SpeedTouch home router that I'm using but 0.21 does seem far more stable for me.

      J.

       
      • JimBob McGee

        JimBob McGee - 2008-03-05

        Actually, I've just found my only complaint with NTFSCLONE -- it is not possible to see the output of the command after a failure; it goes back to the menu too quickly!!

        A beep/bell call and a 10s pause would really help after NTFSCLONE completes...

         
      • JimBob McGee

        JimBob McGee - 2008-03-05

        FYI, BIOS revision A03

         
    • Tecky

      Tecky - 2008-03-05

      I don't know if it can help but,

      I'm using G4L on many model of HP pc.
      The last model that worked find with the default boot switch is
      HP DC7600. After that I started having boot and speed problem.

      With the HP DC7700
      To make it boot properly I had to add pci=config1
      that switch prevent the boot process to stop at acpi discovery.
      and for the speed I had to use hda=noprobe

      The result were from 3mb/s to 25mb/s
      I'm using LZO and i have no problem.

      With the HP DC7800
      To make it boot properly I had to add acpi=off
      and for the speed still have to use hda=noprobe

      Now I'm using the NTFSCLONE to ghost my drive and I can say that
      It take no time compare to Ghost all sectors.

      Tecky ^_^

       
      • JimBob McGee

        JimBob McGee - 2008-03-05

        >> pci=config1
        >> acpi=off
        >> hda=noprobe

        Where do you add these switches? At the boot loader? e.g. bzImageb pci=config1 acpi=off hda=noprobe?

         
        • Tecky

          Tecky - 2008-03-05

          I'm using G4L alpha 7

          In the G4L menu, with the cursor on the version you want
          hit the tab key on your keyboard. It give you the option to
          enter switch that you want to use wan the kernel load.

          If you don't have boot problem. just use hda=noprobe

          God luck

          Tecky

           
          • JimBob McGee

            JimBob McGee - 2008-03-05

            hda=noprobe seems to do the trick with regards to speed -- I can get up to 15MB/s now across my LAN.

            Now all I have to worry about is the 'Remote write failed: Connection reset by peer' message that I get within 20 seconds of the RAW transfer starting.

            Is there an eth0=behave! flag I can specify?!

            J.

             
    • Michael Setzer II

      More messages in one day...

      I'll have to do some looking at these options, and see what the options can do.
      The acpi=off is probable no problem, but will have to look at the others.

      >> pci=config1
      >> acpi=off
      >> hda=noprobe

      Could just add a line to the boot menu with these options, since it would only require a couple of lines in the syslinux.cfg.

      The NTFSCLONE doesn't allow one to put an image on a smaller disk, and there is nothing that I know of to fix this. The only solution is to use the ntfsresize option to reduce the size before doing an image. I have one lab that has all the same systems and drives, but for some reason, the exact same drives have different sizes. In my lab, all except 1 are the same size, but that one is about 25MB smaller, so I use it for the image creation. In the other lab, the machines have about 4 different sizes with the drives, so I just ntfsresized one of the machines to the smallest size, and then make an image. One could resize the image to only the space needed, and then make an image. Then it could be restored to a larger partition, and the use ntfsresize to gain the extra space.

      As for beep/bell, there is a 10 second pause at the end of the NTFSCLONE option, but no beep or message. Probable would be possible if the machine has a speaker, but many new machines don't have them.

      The bzImageb is very old, so don't know what the differences are.

      Also, you use the tab key to be able to modify the boot lines.

       
    • Michael Setzer II

      I need some clarification on some of these options that have been mentioned.

      I could find nothing onpci=config1, but I did find pci=conf1 and pci=conf2-

      It talks about using configuration method 1 or 2, but doesn't explain what it does.

      I've found a 30 page document that talks about the options, but doesn't really explain which ones would or would not be goot do use.

      I also tried teh hda=noprobe, but on my systems this is the disk, and resulted in no drive or partition being seen.
      In the configuration that uses this, is hda present or is it the cd rom drive or what.

      I'm think of adding an option on the menu after teh default option that would already have the pci=conf1 pci=noacpi in addition to the noacpi, that way people wouldn't have to use the tab to add the options, but I need to know which options. These didn't seem to make a difference on my machines, but the hda=noprobe made the hard disk not show up, since that is my drive.

      I have started working on v0.25, and made an alpha1 of it, but it is basically just moving my development system from a machine with Fedora 5 to Fedroa 8, have upgraded all the support programs and libraries to use its files in contrast to the fedora 5. Kernels also goign to be build on that machine now.

      ftp://amd64gcc.dyndns.org/g4l-v0.25alpha1.iso

      ftp://amd64gcc.dyndns.org/g4l-v0.25alpha1.devel.tar.gz

      But I have just gone thru and updated the programs in the /bin, /sbin, /lib, /usr/bin, /usr/sbin, /usr/lib, so the g4l scripts are still the same.

       
    • Michael Setzer II

      Two things that confuse me on this.

      The compressed lzop file is not a compressed archive as is usually done, taking files and folders, and compressing them into an archive. It is a compressed file of disk image, so I'm 100% sure what you get when you uncompress it or where it would get a filename from?

      The second thing, are you using the latest released version of g4l, there was a problem with the ntfsclone on one of the alpha versions, but don't recall which one. I was trying to clean up all the unnecessary libraries, and missed some. I just did a test making an ntfsclone lzop backup, and then restoring it with no problem with the released version.

       
      • JimBob McGee

        JimBob McGee - 2008-03-13

        >>The compressed lzop file is not a compressed archive as is usually done, taking files and folders,
        >>and compressing them into an archive. It is a compressed file of disk image, so I'm 100% sure what
        >>you get when you uncompress it or where it would get a filename from?

        I thought this too; my only guess was that it was doing something clever due to my use of the .lzo extension -- why I would think that, I don't know, but it seemed the only logical reason. I also see it when opening a .gz compressed image -- the filename is reported as the same minus the .gz extension. I've never tried with BZip2.

        I assume this basically works by piping the stdout of dd/ntfsclone to the stdin of gzip/lzop and piping that stdout to the stdin of an ftp client (just a guess -- I'm sure it's more complicated than that :-), so maybe that's just the default behavior of gzip/lzop when dealing with stdin input...?

        >>The second thing, are you using the latest released version of g4l, there was a problem with the
        >>ntfsclone on one of the alpha versions, but don't recall which one. I was trying to clean up all the
        >>unnecessary libraries, and missed some. I just did a test making an ntfsclone lzop backup, and then
        >>restoring it with no problem with the released version.

        I've tried with my old copy of the 0.21 release and the 0.24 release, both in bzImageb and bz22.3. Both fail when restoring lzop files. Maybe it is just due to IIS FTP -- I've tried reading/writing to compressed and uncompressed NTFS folders. I've never had a problem with GZip compression, though...

        I gave up in the end, having tried raw, ntfsclone and click 'n' clone. I used a Norton Ghost 2003 boot disk, which someone had made into a CD ISO (thanks BitTorrent!!) to do my disk-to-disk copy, in the end, which was able to resize the partitions for me on the fly, also.

        A shame really, because I've never had a problem with G4L before...

         
    • Tecky

      Tecky - 2008-03-08

      My mistake Michael, you are right, the option is
      pci=conf1

      I don't know why in my notes I had config1.
      Sorry !

      The reason I'm using pci=conf1 is that my HP DC7700 can't boot without it.
      It hang at probing acpi. And my DC7800 I have to add acpi=off for the same reason.

      For the LZO problem my guess is JimBob should try LZO on Filezilla ftp server.
      It's a small ftp server that can run on any Windows xp PC with enough disk space
      for the image.

      ^_^

       
    • Tecky

      Tecky - 2008-03-12

      Hi Michael,

      I'm testing the new G4L v0.25alpha1 and it's the first time a can boot my
      HP DC7700 and HP DC7800 without adding option pci=conf1
      I'm using the new bz25rc3.gt3 an with that one the boot process doesn't stop at
      probing acpi.

      That was the good news. The bad one now is the option hda=noprobe give me the same
      result than you have. I mean can't find the drive and without that option the
      transfer rate is 4mb.

      Is this new bx25rc3 have been created on your new fedora system ???
      I tried the bz24.3 and it still working fine with the two options.

      Last thing is. the menu keep restarting the counter without opening the default menu
      selection.

      Tecky ^_^

       

Log in to post a comment.