new backup policy...

masavini
2013-06-23
2013-07-04
  • masavini
    masavini
    2013-06-23

    hi,
    my laptop has a 500gb hard disk. to back it up i just bought an IDENTICAL hd (same brand and model) and an external case with usb, then used them to clone the laptop hd with g4l.
    i clone it after testing any change i performed on the operative system or the installed apps, while dropbox incrementally backs up my documents directories.

    it is a wonderful backup system: when it happened the laptop disk got broken (it did), i just physically replaced the internal hd with the clone, booted the system and waited for dropbox to sync the most frequently changing files... in a few minutes i had my full system back, perfectly working and up to date.

    i just bought an asus ux51vz with 2 128gb Adata XM11 ssd drives (with proprietary connectors) in a raid-0 configuration. i will receive it in the next few days and i'm now wondering how to get a similar backup system...

    here are some of the possibilities i considered:

    A) keep the raid-0 configuration and back it up on a single 256gb ssd connected with a usb adapter (http://www.ebay.it/itm/Asus-UX31-UX21-ADATA-XM11-xm11zzb5-sandisk-to-USB-2-0-CABLE-2-5-3-5-SATA-Adapter-/271115393490?pt=US_Drive_Cables_dapters&hash=item3f1fbc99d2).
    doubts:
    1) where to buy a 256gb Adata XM11 ssd?
    2) would a 256gb drive have exactly the same size (sectors number) of 2 128gb raid-0? this is crucial for g4l to properly clone the raid-0 drives...
    3) what does it happen if one of the 128gb ssd drives fails and i physically replace both of them with the 256gb clone? it wouldn't be a raid-0 configuration anymore, shall i expect it to work properly?

    pros:
    - keeping the raid-0 configuration would ensure the highest read/write speed
    - all of the 256gb storage space could be used
    
    cons:
    - many doubts above, would such a system work properly?
    - the 256gb Adata XM11 ssd will surely be pretty expensive
    

    B) break the raid-0 configuration and backup both the ssd drives to another 2 128gb Adata XM11 drives connected with a usb adapter (http://www.ebay.it/itm/Asus-UX31-UX21-ADATA-XM11-xm11zzb5-sandisk-to-USB-2-0-CABLE-2-5-3-5-SATA-Adapter-/271115393490?pt=US_Drive_Cables_dapters&hash=item3f1fbc99d2).
    doubts:
    1) how to break the raid-0? should i expect to find an option in the bios menu?
    2) where to buy the 128gb Adata XM11 ssds for the backup?

    pros:
    - all of the 256gb storage space could be used
    - no g4l issues for drives size
    
    cons:
    - read/write speed would be about half than in raid-0
    - the 2 128gb Adata XM11 ssds will surely be pretty expensive
    - the 2 drives can't be backed up at once
    

    C) break the raid-0 configuration and use one of the 2 drives as backup drive.
    doubts:
    1) how to break the raid-0? should i expect to find an option in the bios menu?
    2) how about the mount process behaviour? after the cloning process the 2 drives will be IDENTICAL (same content, same partition table, even the same UUIDs): there's a concrete risk that ubuntu mounts some partitions from a drive, and some other from the other drive... it would be a mess... how to tell fstab to mount all of the partitions from a single drive?

     pros:
    - nothing to buy
    - no g4l issues for drives size
    
    cons:
    - only 128gb storage space could be used
    - read/write speed would be about half than in raid-0
    

    D) break the raid-0 configuration and set up a raid-1 configuration.
    doubts:
    1) how to break the raid-0 and set-up the raid-1? shall i expect to find an option in the bios menu?

     pros:
    - nothing to buy
    
    cons:
    - only 128gb storage space could be used
    - read/write speed would be about a quarter than in raid-0
    - no chance to get back to a previous backup point if the system gets messed up
    

    as you can see, there are lots of question marks... :)
    if the raid-1 could be set up with a bios option, the D) solution would be the simpliest... but having no restore point would make me very anxious...

    since i only use about 50gb, the C) solution could be the best...
    i guess that the "mount" issue could be solved booting a live cd just after the cloning process, then manually changing the UUID of the backup partitions.

    any comments/hints/warnings?

    thanks

     
  • Raid 0 is not something I like since it offers no protection if either drive fails.
    With Raid 1 you loss have the space since you are mirroring the disks, but for reads you can get the same speed advantage if the OS supports it. Years ago, had a Novell Server with duplexed disk running 3 disks on each controller.

    You don't mention what OS you are running.
    Also, rather than cloning disk that requires an exact or larger disk, you can do images that don't require that. You also don't mention the host OS on the machine.

     
  • masavini
    masavini
    2013-06-24

    hi michael,
    thanks for your reply...

    i agree raid-0 is not a solution... just a potential problem...

    the laptop arrived today, and it runs win 8... i'm planning to delete it and install ubuntu 13.04. i've already tried the live version, and it works like a charm...

    the bios has the option to configure the 2 ssd drives as IDE, AHCI or RAID0. no other option is given...

    i tried and boot g4l, as well, and it sees the 2 drives separately, regardless the SATA configuration set in the bios.

    i know that imaging has several advantages respect to cloning, but i'm not aware enough to mess with mbr (or UEFI!) and stuff like that... cloning is the best solution for me... and i'm pretty convinced that using one of the drives as cloned backup unit, then changing the UUID just after the cloning process (http://ubuntuforums.org/showthread.php?t=1378527) could be a good solution...

     
    • On 24 Jun 2013 at 18:43, masavini wrote:

      To: "[g4l:discussion] " 408762@discussion.g4l.p.re.sf.net
      From: "masavini" masavini@users.sf.net
      Send reply to: "[g4l:discussion] " 408762@discussion.g4l.p.re.sf.net
      Subject: [g4l:discussion] new backup policy...
      Date sent: Mon, 24 Jun 2013 18:43:08 +0000

      hi michael,
      thanks for your reply...
      i agree raid-0 is not a solution... just a potential problem...
      the laptop arrived today, and it runs win 8... i'm planning to delete
      it and install ubuntu 13.04. i've already tried the live version, and
      it works like a charm... the bios has the option to configure the 2
      ssd drives as IDE, AHCI or RAID0. no other option is given... i tried
      and boot g4l, as well, and it sees the 2 drives separately, regardless
      the SATA configuration set in the bios. i know that imaging has
      several advantages respect to cloning, but i'm not aware enough to
      mess with mbr (or UEFI!) and stuff like that... cloning is the best
      solution for me... and i'm pretty convinced that using one of the
      drives as cloned backup unit, then changing the UUID just after the
      cloning process (http://ubuntuforums.org/showthread.php?t=1378527)
      could be a good solution...

      new backup policy...

      It will always see the two physical drives, but depending on how
      the raid is done it may also show an md? devices that is the raid
      drive.

      I generally split disk into multiple partitions to increase the speed
      of making backups. In my classroom, I have a 160G disk, that has
      40G XP, 1G boot, 40G root, and 4G swap. and an extra partition
      were I store images of the others. I can restore the XP in about 12
      minutes from the sda6 partition. The other partitions can also be
      quickly restored or I also keep a set of images and complete disk
      on other servers and external disks.

      Generally, the compressed image is a lot smaller depending on
      use and if the free space is cleared before doing the image.
      My 40G partitions usually compress to about 12G.

      Long ago did a Fedora install on an 40G disk and then did a disk
      image, and it was 12G. Then cleared all the free space and redid
      the image, and it was only 2.5G.

      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/g4l/discussion/408762/
      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

      +----------------------------------------------------------+
      Michael D. Setzer II - Computer Science Instructor
      Guam Community College Computer Center
      mailto:mikes@kuentos.guam.net
      mailto:msetzerii@gmail.com
      http://www.guam.net/home/mikes
      Guam - Where America's Day Begins
      G4L Disk Imaging Project maintainer
      http://sourceforge.net/projects/g4l/
      +----------------------------------------------------------+

      http://setiathome.berkeley.edu (Original)
      Number of Seti Units Returned: 19,471
      Processing time: 32 years, 290 days, 12 hours, 58 minutes
      (Total Hours: 287,489)

      BOINC@HOME CREDITS
      SETI 15178998.277650 | EINSTEIN 11964810.389852
      ROSETTA 7747629.353306 | ABC 16123851.210856

       
  • masavini
    masavini
    2013-06-28

    hi michael,

    i made a lot of attempts with different bios settings and i'm almost done...

    the 2 ssd drives are now in ahci configuration.

    sda is the "main" drive, sdb is the backup. after the cloning process the 2 drives share the same uuids and if i boot ubuntu some partitions are mounted from sda and some other from sdb... a mess...

    i managed and change the uuids of sdb partitions with a live cd and now everything works fine (only sda partitions are mounted at boot) except for one detail: it keeps mounting sdb1 partition at /boot/efi.

    i don't really know if mounting it from sdb instead of sda can be a problem (the system works fine, as i wrote) but i could not find a way to change sdb1 uuid...

    if there's no way to change it or if i should not care if sda1 or sdb1 is mounted at /boot/efi, i'd like to add a couple of lines to g4l script... can the commands uuidgen, tune2fs and mkswap be used in it?

    thanks...

     
    • On 28 Jun 2013 at 21:58, masavini wrote:

      To: "[g4l:discussion] " 408762@discussion.g4l.p.re.sf.net
      From: "masavini" masavini@users.sf.net
      Send reply to: "[g4l:discussion] " 408762@discussion.g4l.p.re.sf.net
      Subject: [g4l:discussion] new backup policy...
      Date sent: Fri, 28 Jun 2013 21:58:58 +0000

      hi michael,

      i made a lot of attempts with different bios settings and i'm
      almost done...

      the 2 ssd drives are now in ahci configuration.

      sda is the "main" drive, sdb is the backup. after the cloning
      process the 2 drives share the same uuids and if i boot ubuntu
      some partitions are mounted from sda and some other from
      sdb... a mess...

      These would leave me to think that you are actually wanting to
      have a mirrored drive setup in which the 1 disk is constantly
      copied to the second disk. This has its pro's and con's. From
      protection from a single disk failing it is good, and is what I
      preferred in servers and duplexing disks was even better. The
      Con's include that it doesn't protect from a major system failure in
      which both drives are corrupted or if a virus or just a user mistake
      causes issues. That is why backups are still important.

      Don't know if ubuntu supports a software or hardware raid.

      I like making image files to external or ftp servers as a backup to
      get systems back up and running quickly.

      i managed and change the uuids of sdb partitions with a live cd
      and now everything works fine (only sda partitions are mounted
      at boot) except for one detail: it keeps mounting sdb1 partition at
      /boot/efi.

      Yes, clones are a bit copy so everything is copied including the
      uuid. Older setups used the disk and partition so it wasn't an
      issue, but not with the systems using uuid they duplication causes
      an issue. Sometimes I will clone a disk to fix one of my classroom
      machines, and the same issue occurs if I reboot without removing
      it.

      i don't really know if mounting it from sdb instead of sda can be
      a problem (the system works fine, as i wrote) but i could not find
      a way to change sdb1 uuid...

      This could be a problem if it isn't 100% the same. Saw a similar
      issue with a mirror setup. If it boots and sees that the disks are not
      in sync, it must figure out which one is the most up-to-date, and
      then rebuild the mirror. On the old Novell servers I had the rebuild
      process would take about 3 hours for a 9G disk, but that was
      233Mhz amd K6-2 server. With the novell it was never a problem,
      but an Admin SCO system had a raid 5 system that had a drive
      fail, and it continued to work fine, but when the replaced the bad
      drive and it went to rebuild it corrupted the how thing. Fortunately,
      they also had a complete tape back to restore to.

      if there's no way to change it or if i should not care if sda1 or
      sdb1 is mounted at /boot/efi, i'd like to add a couple of lines to
      g4l script... can the commands uuidgen, tune2fs and mkswap
      be used in it?

      Again, this is why I like creating image files to an external disk or
      ftp server. I actually do this in class to wipe out a hard disk by
      erasing it, and then do the restore processes during class. Then
      explain How long it would take to rebuild the system from scratch
      without the backup.

      thanks...

      new backup policy...

      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/g4l/discussion/408762/

      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/
      +----------------------------------------------------------+
      Michael D. Setzer II - Computer Science Instructor
      Guam Community College Computer Center
      mailto:mikes@kuentos.guam.net
      mailto:msetzerii@gmail.com
      http://www.guam.net/home/mikes
      Guam - Where America's Day Begins
      G4L Disk Imaging Project maintainer
      http://sourceforge.net/projects/g4l/
      +----------------------------------------------------------+

      http://setiathome.berkeley.edu (Original)
      Number of Seti Units Returned: 19,471
      Processing time: 32 years, 290 days, 12 hours, 58 minutes
      (Total Hours: 287,489)

      BOINC@HOME CREDITS
      SETI 15215481.974287 | EINSTEIN 12042694.599852
      ROSETTA 7786045.803860 | ABC 16123851.210856

       
  • masavini
    masavini
    2013-06-29

    hi michael,

    These would leave me to think that you are actually wanting to have a mirrored drive setup in which the 1 disk is constantly copied to the second disk.

    i don't want the 2 disks to be constantly mirrored: the main purpose of the backup (sdb) is in case the main disk (sda) fails or if something gets messed up after i change some application setting or an update...
    if i wanted a raid-1, i'd create it... :)
    i use dropbox for real time backup, and that's far enough...

    i clone the disk just about once per month, then i perform all of the system updates. i test the new setup for a month, then i clone it again and so on...

    for what i've learned so far, efi system partition is only used at boot, and it's never modified after installation... this means that sda1 and sdb1 will always be the same content and randomly mounting the efi partition from sda or sdb should not be a problem...

    i guess you missed my last question, the only one about g4l... :)

    i'd like to add a couple of lines to g4l script... can the commands uuidgen, tune2fs and mkswap be used in it?

    if i'm not wrong, i remember in older versions of your iso there was a bash script called g4l that could be edited to add some code... i can't find it anymore... how could i add those commands?

     
    • On 29 Jun 2013 at 15:47, masavini wrote:

      To: "[g4l:discussion] " 408762@discussion.g4l.p.re.sf.net
      From: "masavini" masavini@users.sf.net
      Send reply to: "[g4l:discussion] " 408762@discussion.g4l.p.re.sf.net
      Subject: [g4l:discussion] new backup policy...
      Date sent: Sat, 29 Jun 2013 15:47:25 +0000

      hi michael,
      These would leave me to think that you are actually wanting to
      have a mirrored drive setup in which the 1 disk is constantly
      copied to the second disk.
      i don't want the 2 disks to be constantly mirrored: the main purpose
      of the backup (sdb) is in case the main disk (sda) fails or if
      something gets messed up after i change some application setting or an
      update... if i wanted a raid-1, i'd create it... :) i use dropbox for
      real time backup, and that's far enough... i clone the disk just about
      once per month, then i perform all of the system updates. i test the
      new setup for a month, then i clone it again and so on... for what
      i've learned so far, efi system partition is only used at boot, and
      it's never modified after installation... this means that sda1 and
      sdb1 will always be the same content and randomly mounting the efi
      partition from sda or sdb should not be a problem...

      With a clone operation like you are stating, you are using the
      whole second disk to contain a single image that has the issues of
      dup uuids after cloning that would need to be redone each time. It
      you just made that second disk a partition you could probable
      make a number of images on it that could be quickly restored in
      the event of corruption. As I said, I've added g4l as a boot option
      on grub menu, and it can restore to the disk or partitions from the
      image files. The only issue with this is if the sda disk fails, then it
      would have to be replaced, and then image restored.

      i guess you
      missed my last question, the only one about g4l... :)
      i'd like to add a couple of lines to g4l script... can the

      The frontend process adds capabilites to the startup. The g4l
      script is g4l30o11 that could be copied to other distros. I did long
      ago have copies of the g4l script and other statically compliled
      programs.

      commands uuidgen, tune2fs and mkswap be used in it?
      

      uuidgen is only 8000 bytes. Only issue is if it would require
      support libraries that are not already part of the g4l file system.

      tune2fs is 36140 bytes
      mkswap is 24920 bytes.

      Actually, just checked, and tune2fs and mkswap are already on
      the g4l cd, but not direct options on the g4l script. There are a
      number of other programs.

      I have just added uuidgen since it was so small.

      I'm currently building a new alpha with the uuidgen added.

      ftp://amd64gcc.dyndns.org/g4l0.46alpha/g4l-v0.46alpha06.iso

      With Fedora I just use the 40_custom file and put the
      ramdisk.lzma and the default kernel in the boot directory to have it
      as a boot option. Then use grub2-mkconfig to build the grub.cfg
      file.

      It is relatively easy to add commands to the g4l. I am currently
      using fedora 16 as the base system, but have already moved files
      to a fedora 18 machine for testing. A number of programs and
      libraries needed to be updated.

      if i'm not wrong, i remember in older versions of your iso there was a
      bash script called g4l that could be edited to add some code... i
      can't find it anymore... how could i add those commands?

      new backup policy...

      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/g4l/discussion/408762/
      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

      +----------------------------------------------------------+
      Michael D. Setzer II - Computer Science Instructor
      Guam Community College Computer Center
      mailto:mikes@kuentos.guam.net
      mailto:msetzerii@gmail.com
      http://www.guam.net/home/mikes
      Guam - Where America's Day Begins
      G4L Disk Imaging Project maintainer
      http://sourceforge.net/projects/g4l/
      +----------------------------------------------------------+

      http://setiathome.berkeley.edu (Original)
      Number of Seti Units Returned: 19,471
      Processing time: 32 years, 290 days, 12 hours, 58 minutes
      (Total Hours: 287,489)

      BOINC@HOME CREDITS
      SETI 15228662.051047 | EINSTEIN 12069305.599852
      ROSETTA 7798091.544979 | ABC 16123851.210856

       
  • masavini
    masavini
    2013-07-01

    ok, i did my homework... :)

    here is a modified version of the latest downloadable g4l script (g4l30o11.sav319)...

    you can find the modifications by searching for the string "masavini".

    i just added:
    - a new function (uuidChange - line 656)
    - a new choiceclonemenu option ("U: Change UUIDs of cloned ext2, ext3, ext4 and swap partitions" that defines the toggle variable $xuuidChange - line 2064)
    - a call to uuidChange function (line 2123)

    i think this new option could only interest people doing a click 'n clone of their disk...

    what do you think about? could my code be added to the next release? you'd make me happy like child... :)

     
    Attachments
    • On 1 Jul 2013 at 21:25, masavini wrote:

      To: "[g4l:discussion] " 408762@discussion.g4l.p.re.sf.net
      From: "masavini" masavini@users.sf.net
      Send reply to: "[g4l:discussion] " 408762@discussion.g4l.p.re.sf.net
      Subject: [g4l:discussion] new backup policy...
      Date sent: Mon, 01 Jul 2013 21:25:43 +0000

      ok, i did my homework... :)
      here is a modified version of the latest downloadable g4l script
      (g4l30o11.sav319)... you can find the modifications by searching for
      the string "masavini". i just added: - a new function (uuidChange -
      line 656) - a new choiceclonemenu option ("U: Change UUIDs of cloned
      ext2, ext3, ext4 and swap partitions" that defines the toggle variable
      $xuuidChange - line 2064) - a call to uuidChange function (line 2123)
      i think this new option could only interest people doing a click 'n
      clone of their disk... what do you think about? could my code be added
      to the next release? you'd make me happy like child... :) Attachment:
      g4l30o11.masavini (105.0 kB; application/octet-stream)

      Attachment didn't go thru. Need to send it to my direct email.
      mikes@kuentos.guam.net

      new backup policy...

      Sent from sourceforge.net because you indicated interest in
      https://sourceforge.net/p/g4l/discussion/408762/
      To unsubscribe from further messages, please visit
      https://sourceforge.net/auth/subscriptions/

      +----------------------------------------------------------+
      Michael D. Setzer II - Computer Science Instructor
      Guam Community College Computer Center
      mailto:mikes@kuentos.guam.net
      mailto:msetzerii@gmail.com
      http://www.guam.net/home/mikes
      Guam - Where America's Day Begins
      G4L Disk Imaging Project maintainer
      http://sourceforge.net/projects/g4l/
      +----------------------------------------------------------+

      http://setiathome.berkeley.edu (Original)
      Number of Seti Units Returned: 19,471
      Processing time: 32 years, 290 days, 12 hours, 58 minutes
      (Total Hours: 287,489)

      BOINC@HOME CREDITS
      SETI 15254107.222535 | EINSTEIN 12104220.579852
      ROSETTA 7823117.318901 | ABC 16123851.210856

       
  • Will have to take sometime to look at it. Looks like the version you used was from 0.41a, whereas the latest version is 0.45 with changes.

    ftp://amd64gcc.dyndns.org/g4l0.46alpha/g4l30o11

     
  • masavini
    masavini
    2013-07-02

    just sent you an email with an updated version... :)

     
  • I've done some testing as a separate script so I can test it without having to require a clone or change a running system. Have a virtualbox setup that includes two small virtual disks.

    ftp://amd64gcc.dyndns.org/uuid

    This script includes a standalone script for the changing of the uuid as a separate step. It also includes a prompt for the disk to change.

    This will allow it to be added to both the clone mention, and probable the utils menu. You code is great, but this way it can be just added to the menu as an option without having to use multiple functions or requiring a clone.

    I should be able to just pull the one function, and then add it to the clone menu, and then add the call to the function.

     
  • Have created a release that has the option to reset the uuid on a disk as a standalone option on the clone and util menus. It is not on the default script since I am doing testing still. It is run by using /g4l30o11b