Clonezilla / Bugs / #397 clonezilla-live-3.0.2-21 fails dealing with LUKS partitions

Steven Shiau - 2022-12-25

Thanks for this bug report. Could you please let us know how you install Manjaro Linux with two LUKS partitions? We'd like to reproduce this issue here. Thanks.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Swâmi Petaramesh - 2022-12-26
  
  Well that's some relatively simple install using the Manjaro installer with manual partitioning :
  - Create a 256 MB FAT3 EFI partition that mounts at /boot/efi
  - Create a 1 GB ext4 unencrypted partition that mounts at /boot
  - Create a (size depends on your disk) BTRFS partition that will mount as /, check “encrypt”
  - Create a swap partition, chek “encrypt”
  - Install
  
  Given that Clonezilla live will then fail and try to give the same name to LUKS partitions, I tried to make things easier on it by editing /etc/crypttab and replacing UUID=blah partition identifiers by more traditional /dev/nvme0n1p3 and /dev/nvme0n1p4 entries,then regenerate initramfs, but then Clonezilla will state that it is unable to parse crypttab and use random generated names (as it does from the start in Qubes, although Qubes generates the same kind of crypttab that Manjaro does...), and then it works - but uses random names.
  
  If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-01-02

Thanks. I confirmed this issue.Yes, for the moment, Clonezilla only supports 1 LUKS device in a system. As for the mapping name in initramfs, it's crucial to the LUKS because the mapping name has to be same with that in the root file system. An arbitrary name will not work.
We will improve Clonezilla to support more than 1 LUKS device in a system. Please be patient.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-01-10

This issue was fixed in Clonezilla live >= 3.0.3-10 or 20230110:
https://clonezilla.org/downloads.php

Please give it a try and let us know the results.
Thanks.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Swâmi Petaramesh - 2023-01-10

Hello,

I've just tested with clonezilla-live-3.0.3-10-amd64.zip, and unfortunately, the bug is still there.
Clonezilla still tries to open the 2 LUKS volumes giving them the same LUKS name, so the 2nd one fails and clonezilla stops.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-01-10

Thanks for your feedback.
We need more info, so please:
1. Boot Clonezilla live >= 3.0.3-10 amd64, then run it again. Once it's done, keep the log file /var/log/clonezilla.log.
2. Run the command:
ocs-live-ver

Please share the file clonezilla.log and the result of (2).

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Here you go :

root@debian:/# ocs-live-ver
head: cannot open '/run/live/medium/Clonezilla-Live-Version' for reading: No such file or directory
File /run/live/medium/Clonezilla-Live-Version not found.

root@debian:/# find / -name Clonezilla-Live-Version 2>/dev/null
/run/live/medium/live/Clonezilla-Live-Version
/usr/lib/live/mount/medium/live/Clonezilla-Live-Version

root@debian:/# cat /run/live/medium/live/Clonezilla-Live-Version
clonezilla-live-3.0.3-10-amd64
This Clonezilla live zip file was created by this command:
ocs-live-dev -c -s --extra-boot-param quiet loglevel=0 -y 6.03 -i 3.0.3-10-amd64

root@debian:/# cat /usr/lib/live/mount/medium/live/Clonezilla-Live-Version
clonezilla-live-3.0.3-10-amd64
This Clonezilla live zip file was created by this command:
ocs-live-dev -c -s --extra-boot-param quiet loglevel=0 -y 6.03 -i 3.0.3-10-amd64

Log file :

Starting /usr/sbin/ocs-sr at 2023-01-11 08:03:49 UTC...
*****************************************************.
Clonezilla image dir: /home/partimag
Shutting down the Logical Volume Manager
Finished Shutting down the Logical Volume Manager
The selected devices: nvme0n1
PS. Next time you can run this command directly:
/usr/sbin/ocs-sr -luks yes -q2 -c -j2 -gs -z2p -i 4096 -sfsck -senc -p choose savedisk vger_W11_Mjro_230111b nvme0n1
*****************************************************.
The selected devices: nvme0n1
Searching for data/swap/extended partition(s)...
The data partition to be saved: nvme0n1p1 nvme0n1p2 nvme0n1p3 nvme0n1p4 nvme0n1p5 nvme0n1p6 nvme0n1p7 nvme0n1p8
The selected devices: nvme0n1p1 nvme0n1p2 nvme0n1p3 nvme0n1p4 nvme0n1p5 nvme0n1p6 nvme0n1p7 nvme0n1p8
The following step is to save the hard disk/partition(s) on this machine as an image:
*****************************************************.
Machine: VivoBook_ASUSLaptop X415JAB_X415JA
nvme0n1 (2000GB_CT2000P2SSD8__CT2000P2SSD8_2131E5BDDB5F)
nvme0n1p1 (260M_vfat_SYSTEM(In_CT2000P2SSD8_)_CT2000P2SSD8_2131E5BDDB5F)
nvme0n1p2 (16M_MS_Reserved_Partition(In_CT2000P2SSD8_)_CT2000P2SSD8_2131E5BDDB5F)
nvme0n1p3 (236G_Basic(In_CT2000P2SSD8_)_CT2000P2SSD8_2131E5BDDB5F)
nvme0n1p4 (1000M_ntfs_RECOVERY(In_CT2000P2SSD8_)_CT2000P2SSD8_2131E5BDDB5F)
nvme0n1p5 (200M_vfat_MYASUS(In_CT2000P2SSD8_)_CT2000P2SSD8_2131E5BDDB5F)
nvme0n1p6 (1G_ext4(In_CT2000P2SSD8_)_CT2000P2SSD8_2131E5BDDB5F)
nvme0n1p7 (1.6T_crypto_LUKS(In_CT2000P2SSD8_)_CT2000P2SSD8_2131E5BDDB5F)
nvme0n1p8 (12G_crypto_LUKS(In_CT2000P2SSD8_)_CT2000P2SSD8_2131E5BDDB5F)
*****************************************************.
-> "/home/partimag/vger_W11_Mjro_230111b".
Shutting down the Logical Volume Manager
Finished Shutting down the Logical Volume Manager
*****************************************************.
Now open the LUKS device: /dev/nvme0n1p7
Running: cryptsetup open --type luks /dev/nvme0n1p7 luks-crypttab-from-OS.info:luks-fbeb41fe-6b55-4bcd-9479-1be708e43b09
*****************************************************.
Now open the LUKS device: /dev/nvme0n1p8
Running: cryptsetup open --type luks /dev/nvme0n1p8 luks-crypttab-from-OS.info:luks-fbeb41fe-6b55-4bcd-9479-1be708e43b09

luks-crypttab-from-OS.info

# /etc/crypttab: mappings for encrypted partitions.
#
# Each mapped device will be created in /dev/mapper, so your /etc/fstab
# should use the /dev/mapper/<name> paths for encrypted devices.
#
# See crypttab(5) for the supported syntax.
#
# NOTE: Do not list your root (/) partition here, it must be set up
#       beforehand by the initramfs (/etc/mkinitcpio.conf). The same applies
#       to encrypted swap, which should be set up with mkinitcpio-openswap
#       for resume support.
#
# <name>               <device>                         <password> <options>
luks-62de0a8e-145c-4343-ab48-cd719c4c55a2 UUID=62de0a8e-145c-4343-ab48-cd719c4c55a2     none luks,discard
luks-fbeb41fe-6b55-4bcd-9479-1be708e43b09 UUID=fbeb41fe-6b55-4bcd-9479-1be708e43b09     none luks,discard

Steven Shiau - 2023-01-11

Did you modify the Clonezilla live zip file? Otherwise it should have a file "Clonezilla-Live-Version" in the root dir.
$ unzip -l clonezilla-live-3.0.3-10-amd64.zip | grep Clonezilla-Live-Version
171 2023-01-10 11:37 live/Clonezilla-Live-Version
171 2023-01-10 11:37 Clonezilla-Live-Version

I addition, one thing I do not understand is, why here it showed:
Running: cryptsetup open --type luks /dev/nvme0n1p7 luks-crypttab-from-OS.info:luks-fbeb41fe-6b55-4bcd-9479-1be708e43b09
Why it contains the file name "luks-crypttab-from-OS.info" + ":"?

Here what I have is:
********.
Found LUKS device: nvme0n1p3 nvme0n1p4
We have to open LUKS device now. You will be prompted to enter passphrase for ea
ch one of them.
Extracting initramfs: /tmp/ird_mnt.vpV/initramfs-6.1-x86_64.img... done.
Not any LUKS crypttab file was found inside initramfs file.
Running: cryptsetup luksHeaderBackup --header-backup-file /home/partimag/2023-01
-11-16-img/nvme0n1p3-luksHeader.bin /dev/nvme0n1p3
*******.
Now open the LUKS device: /dev/nvme0n1p3
Running: cryptsetup open --type luks /dev/nvme0n1p3 ocs_luks_dua
Enter passphrase for /dev/nvme0n1p3:
Running: cryptsetup luksHeaderBackup --header-backup-file /home/partimag/2023-01-11-16-img/nvme0n1p4-luksHeader.bin /dev/nvme0n1p4
*******.
Now open the LUKS device: /dev/nvme0n1p4
Running: cryptsetup open --type luks /dev/nvme0n1p4 ocs_luks_0v0
Enter passphrase for /dev/nvme0n1p4:
********.

Could you please tell me the whole procedure, step by step, from how you put Clonezilla live 3.0.3-10 amd64 zip in your USB stick, and how to boot it, choose which one in the boot menu, how you run to save the LUKS devices... I'd like to reproduce this issue here.
Thank you very much.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-01-11

BTW, if you are familiar with screen or tmux, please enter it, then run:
bash -x /usr/sbin/ocs-sr -luks yes -q2 -c -j2 -gs -z2p -i 4096 -sfsck -senc -p choose savedisk vger_W11_Mjro_230111b nvme0n1

then copy all the outputs on the screen and paste to a text file. Then share it.
Thanks.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Swâmi Petaramesh - 2023-01-11

Hi,

No I didn't modify the live zip file in any way, I just extracted it to an empty USB drive per Clonezilla's website instructions for creating an EFI bootable USB key.

So the whole procedure (in KDE Plasma) :
- Insert the USB drive, reformat it blank.
- Right-click on the Clonezilla zip file in Dolphin, select “extract to”, point to the usb drive, that's all about it.
- Equivalent to just unzip on the USB drive.

I booted it on an EFI machine using the 2nd Clonezilla boot menu option (running from RAM)

No it dosesn't output “luks-crypttab-from-OS.info”, It's just the contents of my luks-crypttab-from-OS.info (generated by Clonezilla) that I attached to the bug report in case it might help - I thought the bug report layout would make it clear.

Per your request :

❯ unzip -l clonezilla-live-3.0.3-10-amd64.zip | grep Clonezilla-Live-Version 171 2023-01-10 04:37 live/Clonezilla-Live-Version 171 2023-01-10 04:37 Clonezilla-Live-Version

(Shown time difference with your example is probably just because we don't live in the same time zone...)

I'll try the test you ask me for and join the output when done.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Swâmi Petaramesh - 2023-01-11

Here is the output you requested (compressed as it is rather verbose...)

script_230111c.log.xz

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Swâmi Petaramesh - 2023-01-11

Per what I understand from the logs, it looks like there is some misconstruct in the “grep -Ew | sort | awk | uniq | head” (Uuuuuh...) that Clonezilla uses to get the LUKS device name from luks-crypttab-from-OS.info (missing quotes or whatever) ? which ultimately results in Clonezilla getting for the 1st LUKS device the LUKS name of the 2nd one and open it with this name.

Then it gets the same LUKS name for the 2nd device and tries to open it with the same LUKS name and boom.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-01-11

Yes, I also found where the bug is. However, the Manjaro 22 I tested does not have crypttab in initramfs, though I have 2 LUKS devices. Therefore I can not test if the fixed ocs-functions works or not.
If you can, please modify /usr/share/drbl/sbin/ocs-functions, the line 16152, append "| uniq)" in the end like:

search_pat="$(grep -Eo "^[^#].*[[:space:]]+(UUID=|LABEL=|PARTUUID=|PARTLABEL=)" \ $img_full_path/luks-crypttab-from-OS.info | awk -F" " '{print $2}' | sed -r -e "s/=$//g" | uniq)"

then run "sudo clonezilla" to test it again.
Please let me know the results. Thanks.

Steven

👍
1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-01-11

BTW, just curious, how did you install your Manjaro?
I followed what you have mentioned, in a VM, I installed with manjaro-xfce-22.0-221224-linux61.iso, and manually make it have partition layout like:

Model: VMware Virtual NVMe Disk (nvme) Disk /dev/nvme0n1: 21.5GB Sector size (logical/physical): 512B/512B Partition Table: gpt Disk Flags: Number Start End Size File system Name Flags 2 1049kB 269MB 268MB fat32 boot, esp 1 269MB 1343MB 1074MB ext4 3 1343MB 20.4GB 19.1GB 4 20.4GB 21.5GB 1049MB

Partition 3 and 4 are LUKS devices. I followed the way you mentioned to install it. Actually I am not familiar with Arch/Manjaro...

Steven
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Swâmi Petaramesh - 2023-01-11

I'll test your suggestion and let you know (I'm not sure whether booting Clonezilla live in RAM will allow me to modify the script) ?

About the way I installed Manjaro, well it dit it quite like you did, except that the machine already had Windows on it, so that's a dual boot setup (more partitions...).

But I forgot one point, it's that post-installation I changed from to busybox standard based initrd to a systemd based initrd (and also removed grub in favor of rEFInd).

Basically to do this you will remove the “cryptdevice” stanzas from /etc/default grub, and instead the crypttab will be included into the initramfs.

What you need to do for this conversion is a bit complicated (but not too much) :
- copy your /etc/crypttab to /etc/crypttab.initramfs (you may remove from it additional volumes that wouldn't be needed to boot the machines, only keep / and swap if needed for hibernation. Of course then Clonezilla could not get info about them by looking into initramfs' crypttab).
- Edit your /etc/mkinitcpio.conf and replace busybox hooks with systemd hooks (ArchLinux wiki says everything about it : https://wiki.archlinux.org/title/mkinitcpio#HOOKS )
- Rebuild the initramfs with : mkinitcpio -P
- Edit /etc/defautlt/grub and replace entries needed for a busybox initramfs by entries needed for a systemd initramfs, then update-grub.

Well that's a bit complex but I had completely forgotten that you don't get any crypttab in the initramfs if you don't do this (unless you hack the mkinitcpio.conf to copy it over anyway).

But I assume that the issue we have with parsing crypttab in clonezilla would be the same in every other distro that copies it over to the initramfs as soon as you have several LUKS filesystems... So probably vanilla Fedora or vanilla Debian would do the same (as I have the same issue with a vanilla Qubes...)

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Swâmi Petaramesh - 2023-01-11

I'm happy to report that the bug fix you suggest in your comment at https://sourceforge.net/p/clonezilla/bugs/397/#c603 seems to fix the issue here.

At least Clonezilla was able to open the 2 LUKS devices giving them both the correct name.

(Now it's in process of cloning my machine so it will takes hours until I can say if everything goes well up to the end and clone checks later on, but so far, so good...)

Thank you very much :)

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-01-14

OK, great. Thanks for your help.
We will improve this mechanism to meet more formats in crypttab. The next testing release will be released asap.
Thank you very much.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-01-21

If you have chances, please give Clonezilla live 3.0.3-12 or 20230118-* a try:
https://clonezilla.org/downloads.php
They are improved about multiple LUKS devices.
Please let us know the results if you test it. Thanks.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Swâmi Petaramesh - 2023-01-23

Hello,
I'm happy to report that it works good on Manjaro.
Thank you very much for your help !
Kind regards.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Swâmi Petaramesh - 2023-01-24

Uh sorry, I spoke too fast.

There's something really weird going on.

My machine has a 1 TB SSD, most of which is a ~900 GB encrypted LUKS partition, of which about half is used (ie. 450 GB used, 450 GB free).

When I used Clonezilla 3.0.3-10 with the fix you proposed, it seemed to work and I ended up with a XZ clone totalling 160 GB, which I would expect for a correct processing of the (unencrypted) filesystem contents, with XZ compression. Seemed good.

Now that per your request I used 3.0.3-12 on the same machine, it appeared to work, but I ended up with a 471 GB clone - Yes this is about the actually used size on the FS. Which makes me think that either the encrypted (uncompressible) contents were actually saved instead of the decrypted ones, or the XZ compression didn't work - event though my files do have the “.xz.aa” suffixes.

But surely, something went wrong (even though Clonezilla told it had cheked the saved image OK).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Steven Shiau - 2023-01-27

Since we have done a lot of changes from Clonezilla live 3.0.3-12, we'd better to sync the version.
Please give Clonezilla live 3.0.3-20 a try. If this issue remains, let us know the details about the steps to reproduce it. If it's reproducible here, then we will do our best to fix it.
Thanks.

Steven

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

clonezilla-live-3.0.2-21 fails dealing with LUKS partitions

A partition and disk imaging/cloning program

Group

Searches

Help

#397 clonezilla-live-3.0.2-21 fails dealing with LUKS partitions

Discussion

Log file :

luks-crypttab-from-OS.info