Clonezilla / Discussion / Clonezilla live: CA dry-run mode (suggestion)

mildred ratched - 2023-11-10

Hi Steve,

I was wondering if it were possible to add a dry-run mode to Clonezilla where it would issue all the commands/actions it would make, without however making any change to the target partition/device?

This could save a huge amount of time in debugging. I don't have any spare HDD anymore so I need to take a backup HDD, overwrite it with ZA, test it, and indepedently if the process was successful or not, put the backup back again in place. Not to mention the time to nake the first backup, prior to use it for CZ testing.

Usually I use a several year old parted magic live CDor gparted 0.15, both from 2013 for my backup/restore operations because it's been proven best to use the same version of CZ to restore a backup made from CZ. I've seen where not doing this guarantees failure from ever booting again.

More recently, I started to use Luks for Linux whereas I used Truecrypt 7.1a for windows disks/partitions. I know how to use partimage from the command line to backup/restore the decrypted versions of them and reencrypt the target partition once done, restoring the original uuids but it's not very useful when the disk/partitions are almost full: it takes more time than dd!

I have some new installs though with only a few GB in 500GB partitions, and I could benefit from such a dry-run feature to debug. Decrypting a few GB of data, rsyncing them to the target and re-encrypting the target and recopying the uuids over (for grub/fstab/cryptab/inittab) would be a lot faster than dding a 500GB HD/partition, wouldn't it? I tested 2 systems (Debian 12 and Fedora 38) with the latest CZ, but it silently falls back to dd mode for my luks partitions, all of them. Having no HDD anymore, I can't test.

I'm pretty sure you understand.

Best!

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-11-12

Hi,
Thanks for your feedback.
The most related dry-run function in Clonezilla now is to verify the image.
We do not have any mode for test partition table creation, and that of LV, LUKS...
Have you used Clonezilla live to save and restore your LUKS mode? Did you encounter any issue?

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

mildred ratched - 2023-11-12

I see. No problem. Just wondering.

Have you used Clonezilla live to save and restore your LUKS mode? Did you encounter any issue?

Oh yes, save never worked on Debian 12 nor Fedora 38:
https://sourceforge.net/p/clonezilla/discussion/Clonezilla_live/thread/169b566ba8/#ec61
https://sourceforge.net/p/clonezilla/discussion/Clonezilla_live/thread/8d07da1b35/

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-11-12

Is that possible you can install your Debian 12 or Fedora 38 on a virtual machine, then share the VM with us so that we can try to reproduce this LUKS saving issue?

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

mildred ratched - 2023-11-12

Hi Steven,

Unrealistic. Those are 500GB each. If you can explain how to turn on verbose debugging information to be logged while saving, I could share the logs with you. Current logs are useless: they contain zero clue about why, how or when CZ choose to ignore crypto_LUKS partitions he did detect and switch silently to dd mode. Makes sense?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-11-12

Or maybe we start from Deban 12 first. How did you install that with LUKS enabled? Please explain in detail so that I can install here and try to reproduce this issue.
Thanks.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

I did this years ago, possibly with Debian 6 or 7. I manually created the partitions or I used gparted, not sure. After upgrading to Debian 8, I encrypted partitions 2, 3 and 7 with luks, manually, from the command line. I continued to upgrade Debian throughout the years, Debian 9, 10, 11, then 12.

Recently, I cloned my HDD to an NVMe SSD so I thought it would be time to realign the partitions properly:

Before:

                                                            Disque : /dev/nvme0n1
                                             Taille : 465,76 GiB, 500107862016 octets, 976773168 secteurs
                                                      Étiquette : dos, identifiant : 0xedaaedaa

    Périphérique           Amorçage                 Début             Fin        Secteurs      Taille      Id Type
>>  /dev/nvme0n1p1         *                           63         6146047         6145985        2,9G      83 Linux
    /dev/nvme0n1p2                                6146048        66347007        60200960       28,7G      83 Linux
    /dev/nvme0n1p3                               66348450       308744191       242395742      115,6G      83 Linux
    /dev/nvme0n1p4                              308744192       976769023       668024832      318,5G       5 Étendue
    ├─/dev/nvme0n1p5                            308746240       316551167         7804928        3,7G      82 partition d'échange Linux / Solaris
    ├─/dev/nvme0n1p6                            316553216       770990079       454436864      216,7G      83 Linux
    └─/dev/nvme0n1p7                            770991543       976768064       205776522       98,1G      83 Linux
    Espace libre                                976769024       976773167            4144          2M


After:
                                                            Disque : /dev/nvme0n1
                                             Taille : 465,76 GiB, 500107862016 octets, 976773168 secteurs
                                                      Étiquette : dos, identifiant : 0xedaaedaa

    Périphérique           Amorçage                 Début             Fin        Secteurs      Taille      Id Type
>>  /dev/nvme0n1p1         *                         2048          6143999         6145951        2,9G      83 Linux
    /dev/nvme0n1p2                                6144000         66346401        60202401       28,7G      83 Linux
    /dev/nvme0n1p3                               66346402        308742143       242395741      115,6G      83 Linux
    /dev/nvme0n1p4                              308742144        976767935       668025791      318,5G       5 Étendue
    ├─/dev/nvme0n1p5                            308744192       316553215         7809023        3,7G      82 partition d'échange Linux / Solaris
    ├─/dev/nvme0n1p6                            316553216       770990079       454436863      216,7G      83 Linux
    └─/dev/nvme0n1p7                            770990080       976769023       205778943       98,1G      83 Linux
    Espace libre                                976769024       976773167            4143          2M

Partition 1 is boot.
Partition 3 is my root partition.
Partition 5 is swap.
Partition 6 is xfs, others are ext4.
Partitions 2, 3, and 7 are luks encrypted.

Here, I first cloned my nvme drive to an HDD sdb using "sudo sh -c 'pv < /dev/nvme0n1 > /dev/sdb'"

Then cloning back was not option anymore because partitions sizes changed so I proceeded to recreate the partitions on the nvme drive and copy the data back from sdb to nvme0n1:

# Delete all existing partitions on the disk
sudo sfdisk --delete /dev/nvme0n1

# Create the partition layout
echo 'label: dos
/dev/nvme0n1p1 : start=2048, size=6145951, type=83
/dev/nvme0n1p2 : start=6144000, size=60202401, type=83
/dev/nvme0n1p3 : start=66346402, size=242395741, type=83
/dev/nvme0n1p4 : start=308742144, size=668025791, type=5
/dev/nvme0n1p5 : start=308744192, size=7809023, type=82
/dev/nvme0n1p6 : start=316553216, size=454436863, type=83
/dev/nvme0n1p7 : start=770990080, size=205778943, type=83' | sfdisk /dev/nvme0n1

# Set the first partition as active
parted /dev/nvme0n1 set 1 boot on


# Retrieve the features of the file systems and create the new partitions
for partition in $(lsblk -e7 -o NAME | grep nvme0n1p | awk '{print $1}'); do
    if [[ $partition == *"nvme0n1p1"* || $partition == *"nvme0n1p2"* || $partition == *"nvme0n1p3"* || $partition == *"nvme0n1p7"* ]]; then
        features=$(dumpe2fs /dev/${partition} | grep 'Filesystem features'|sed 's/,needs_recovery//'| awk '{print $2}')
        mkfs.ext4 -O $features /dev/${partition}
    elif [[ $partition == *"nvme0n1p6"* ]]; then
        mkfs.xfs -f -i size=2048 /dev/${partition}
    fi
done

# Luks encrypt partitions 2, 3 and 7
for partition in nvme0n1p2 nvme0n1p3 nvme0n1p7; do
    # Encrypt the new partitions
    cryptsetup luksFormat --type luks2 --align-payload=32768 /dev/${partition}
    # Format the new encrypted partitions
    cryptsetup open /dev/${partition} ${partition}_crypt
    mkfs.ext4 /dev/mapper/${partition}_crypt
    # Mount the new encrypted partitions
    mkdir /mnt/${partition}
    mount /dev/mapper/${partition}_crypt /mnt/${partition}
done

# Mount the source partitions
for partition in $(lsblk -e7 -o NAME | grep sdb | awk '{print $1}'); do
    if [[ $partition == *"sdb2"* || $partition == *"sdb3"* || $partition == *"sdb7"* ]]; then
        cryptsetup open /dev/${partition} ${partition}_crypt
        mkdir /mnt/${partition}
        mount /dev/mapper/${partition}_crypt /mnt/${partition}
    elif [[ $partition == *"sdb1"* || $partition == *"sdb6"* ]]; then
        mkdir /mnt/${partition}
        mount /dev/${partition} /mnt/${partition}
    fi
done


# Copy the UUIDs from the source partitions to the destination partitions
for partition in $(lsblk -e7 -o NAME | grep sdb | awk '{print $1}'); do
    if [[ $partition != *"sdb4"* ]]; then
        # Get the UUID of the source partition
        uuid=$(blkid /dev/${partition} | awk -F '"' '{print $2}')
        # Set the UUID on the destination partition
        if [[ $partition == *"sdb2"* || $partition == *"sdb3"* || $partition == *"sdb7"* ]]; then
            cryptsetup luksUUID --uuid=$uuid /dev/nvme0n1p${partition#*b}
        else
            tune2fs -U $uuid /dev/nvme0n1p${partition#*b}
        fi
    fi
done


# Copy the data from the source partitions to the destination partitions
for partition in $(lsblk -e7 -o NAME | grep sdb | awk '{print $1}'); do
    if [[ $partition != *"sdb4"* &&  $partition != *"sdb5"* ]]; then
        rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} /mnt/${partition}/ /mnt/nvme0n1p${partition#*b}
    fi
done

# Unmount the source partitions
for partition in $(lsblk -e7 -o NAME | grep sdb | awk '{print $1}'); do
    if [[ $partition != *"sdb4"* &&  $partition != *"sdb5"* ]]; then
        umount /mnt/${partition}/
    fi
    if [[ $partition == *"sdb2"* || $partition == *"sdb3"* || $partition == *"sdb7"* ]]; then
        cryptsetup close ${partition}_crypt
    fi
done

# Unmount the destination partitions
for partition in $(lsblk -e7 -o NAME | grep nvme0n1p | awk '{print $1}'); do
    if [[ $partition != *"nvme0n1p4"* &&  $partition != *"nvme0n1p5"* ]]; then
        umount /mnt/${partition}/
    fi
    if [[ $partition == *"nvme0n1p2"* || $partition == *"nvme0n1p3"* || $partition == *"nvme0n1p7"* ]]; then
        cryptsetup close ${partition}_crypt
    fi
done


# Add keys for auto mount at boot
sudo cryptsetup luksAddKey /dev/nvme0n1p2 /etc/keys/sda2_crypt
sudo cryptsetup luksAddKey /dev/nvme0n1p7 /etc/keys/sda7_crypt

Steven Shiau - 2023-11-14

OK, thanks.
So what were the error messages when you saved your LUKS devices using Clonezilla live 3.1.1-27?
If you can, please share the log file /var/log/clonezilla.log.
Thanks.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

mildred ratched - 2023-11-14

https://sourceforge.net/p/clonezilla/discussion/Clonezilla_live/thread/8d07da1b35/#2865

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

CA dry-run mode (suggestion)

A partition and disk imaging/cloning program

Forums

Help

CA dry-run mode (suggestion)

CA dry-run mode (suggestion)

A partition and disk imaging/cloning program

Forums

Help

CA dry-run mode (suggestion) document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

CA dry-run mode (suggestion)