Clone Loop

[goose]

I am attempting to use CloneZilla to migrate my Debian installation from
one device to another.

The original device which has Debian, /dev/sda, is a 512 GB SATA SSD.

The new device which I want to clone Debian to, /dev/nvme3n1p, is a 512 GB
NVMe SSD.

I have 3 partitions on my SATA SSD:

• /dev/sda1 - /boot
• /dev/sda2 - /boot/efi
• /dev/sda3 - LVM with 1 PV (physical volume)/VG (volume group), and 6 LV's
  (logical volumes) in that one VG.

Prior to this clone, I shrunk the PV and the actual partition of /dev/sda3
to 460 GB, to ensure there wouldn't be any issues in case the new drive was
just a few bytes smaller than the old one.

My CloneZilla seems to be in a "loop", when it completes cloning the
/dev/sda3 partition, it then starts itself over.

1 minute video: https://www.youtube.com/watch?v=URHAbJmQAX8

Would appreciate any advice on how to fix this, so I can get this system
back online!

[goose]

Additional video. This time I loaded CloneZilla to RAM, and then also added the option to run fsck -y. Same result - https://www.youtube.com/watch?v=gBVj240aEj0

[Steven Shiau]

So this is a disk to disk cloning, and the source disk contains some LVs.
Which version of Clonezilla live did you use? Have you tried the latest one, e.g. 2.7.1-22 or even testing ono, i.e., >= 2.7.2-12?
https://clonezilla.org/downloads.php

Steven

[Matt]

I had this exact same issue today. Copying from an internal 250GB NVMe to a new 1TB NVMe drive in an enclosure. I installed Clonezilla 2.7.1-22 onto the new drive then booted using the "To RAM" method. All seemed fine but it loops on /dev/nvme0n1p3 -> /dev/sda3 (which has LVM). I tried it with and without fsck first - same issue. No errors that I could see, just looping.

I did a slow-mo video of the messaging on the screen between each time it looped, and the only notable line says:

/dev/sda3: 8 bytes were erased at offset 0x00000218 (LVM2_member): 4c 56 4d 32 20 30 30 31

Any ideas?

My system is also Debian (Proxmox to be specific).

[Matt]

Digging through some more threads, it looks like the issue is the LVM thin provisioning that Proxmox uses. Not sure if this is the same setup you're dealing with @goose?

It looks like I'll have to find another way.

[goose]

I'm out and about, so I can't type a lengthy reply right now. I managed to get this fixed by doing the clone manually. See this reddit comment I posted: https://www.reddit.com/r/homelab/comments/maumz0/z/gs1zws6?context=99999

[Steven Shiau]

I can not reproduce this issue using Clonezilla live 2.7.1-22 to clone a CentOS 7 from sda to nvme0n1. I was testing in a VM on VMWare Workstation 16. Maybe it's because that my sda does not have thin provisioning LVM?
Attached please find the whole process I have done in Clonezilla live.

[breakaway]

I just had the same issue as @goose trying to clone a 2.5" 480 GB SSD (sda) to a 800 GB 2.5" SSD (sdc)

I used clonezilla-live-3.0.1-8-amd64 with toram option to boot (this is what I always use, I've cloned hundreds of machines without issue)

The source disk was a proxmox boot volume, i.e. the OS itself, and a bunch of VMs stored as lvm-thin in the "local-lvm" storage (for anyone familiar with proxmox).

After the second time it copied ~500G, I just hit CTRL+C after it showed "cloned sucessfully." I then shut down the system, swapped the "Target" disk into it, and voila -- everything works perfectly.

This must be something to do with thin provisioning as @steven_shiau says. So this is the hacky workaround -- just kill the process after the clone finishes before the next "round" of cloning starts.

I suspect if I'd just left it it'd complete one round of clone for each LVM volume that's on there (Which is over 10, so it'd take 10x as long).

So steps to reproduce

1. Install latest version of Proxmox
2. Setup a thin provisioned VM (any OS, Debian is OK) on the "local-lvm" storage
3. Clone this vm 2 or 3 time so you have a total of 3 or 4 disks
4. Clone this this entire system to another blank disk.

[Steven Shiau]

Thanks for sharing that.
You mentioned LVM thin provisioning, actually Clonezilla does not support it well. It might be the culprit.

Steven

[ccedricc]

Same problem.

Clonezilla stable - 3.1.0-22, (clonezilla-live-3.1.0-22-i686.iso)
Beginner mode, Device -> Device
A Debian source SSD (1To) and a target HDD (2 To).
The source contain 4 partitions, with on the last partition, a lvm volume group (at least 15 logical volumes inside).

Clonezilla is in loop on this partition since yesterday 22h.
2h10 each loop with an old SATA 2 computer dedicated to this operation.

Thanks @breakaway, i will try your hack workaround CTRL-C at then end of the pass.

Partclone v0.3.23
Starting to clone/restore (/dev/sdb4) to (/dev/sda4) with dd mode
Calculating bitmap... Please wait ... done !
File system: raw
....

[Steven Shiau]

If you can take an image of the source machine, and share that with us. We will try to reproduce this issue here.
Surely you should remove classified/sensitive data from the machine before you take an image.

Steven

[ccedricc]

Thank you for your support.

I make some news experiences and i reproduce this bug with small disks.

All experiences are with Clonezilla stable - 3.1.0-22, (clonezilla-live-3.1.0-22-i686.iso)
Beginner mode, Device -> Device, Disk -> Disk, all defaults options

For this two experiences, i had all : log files, dd image

I use this command to make the images of 2Go and 8Go disks :
dd if=/dev/sdx of=/hdd/first_and_second.img status=progress

The first experience :
I make a "basic" installation on a 2G USB key, with 3 LVMs volumes (more information below). Clonzilla make 2 pass to clone the LVM partition.

The second experience :
I make a "basic" installation on a 8G USB key, with 8 LVMs volumes (more information below). Clonzilla make 7 pass to clone the LVM partition.

The 2G experience :
Debian 12 installation
1) /boot/efi (50M)
2) /boot (150M, ext4)
3) LVM group vg01 with the whole free disk
-> swap (50M)
-> home (50M, ext4, /home)
-> root (the rest, ext4, /)

The source USB (2Go) and a target HDD (160 Go) :
It make 2 pass with dd mode (Device size : 1.8 GB = 3555328 ...)
It make all the ending process.

The 8G experience :
Debian 12 installation
1) /boot/efi (50M)
2) /boot (150M, ext4)
3) LVM group vg01 with the whole free disk
-> root (400M, ext4, /)
-> usr (2G, ext4, /usr )
-> var (1G, ext4, /var)
-> swap (500M)
-> usrlocal (500M, ext4, /usr/local)
-> srv (1G, ext4, /srv)
-> home (1G, ext4, /home)
-> box (the rest, ext4, /box)

The source USB (8Go) and a target HDD (160 Go) :
It make 7 pass with dd mode (Device size : 7.8 GB = 15253504 ...)
It make all the ending process.

I only have an ADSL connection.
If you want, i can split the 8G image (i read some instructions on the WEB to do this) and send you the files (with instructions to get the original), through a web transfer site that limit the maximum file size.

Below the Clonezilla logs files.

Thanks to you and your team.

[Steven Shiau]

"If you want, i can split the 8G image (i read some instructions on the WEB to do this) and send you the files (with instructions to get the original), through a web transfer site that limit the maximum file size. " -> Yes, that would be very helpful.
Here I just can not reproduce this issue. All my LVM devices can be cloned to another disk without duplication actions.
The source devices on /dev/nvme0n1:

root@debian:~# pvs
  PV             VG Fmt  Attr PSize  PFree
  /dev/nvme0n1p3 cl lvm2 a--  18.41g    0
root@debian:~# vgs
  VG #PV #LV #SN Attr   VSize  VFree
  cl   1   2   0 wz--n- 18.41g    0
root@debian:~# lvs
  LV   VG Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  root cl -wi------- 16.41g
  swap cl -wi-------  2.00g
  ~~~

The log for cloning /dev/nvme0n1 to /dev/sda, grep the action, it's like:

root@debian:~# grep -E "^Running: partclone" /var/log/clonezilla.log
Running: partclone.vfat -z 10485760 -L /var/log/partclone.log -b -s /dev/nvme0n1p1 -O /dev/sda1
Running: partclone.ext4 -z 10485760 -L /var/log/partclone.log -b -s /dev/nvme0n1p2 -O /dev/sda2
Running: partclone.dd -z 10485760 -L /var/log/partclone.log -s /dev/nvme0n1p3 -O /dev/sda3

As you can see, no duplicated actions.
While your log appears very different results:

G$ grep -E "^Running: partclone" clonezilla.log
Running: partclone.vfat -z 10485760 -N -L /var/log/partclone.log -b -s /dev/sdc1 -O /dev/sdb1
Running: partclone.ext4 -z 10485760 -N -L /var/log/partclone.log -b -s /dev/sdc2 -O /dev/sdb2
Running: partclone.dd -z 10485760 -N -L /var/log/partclone.log -s /dev/sdc3 -O /dev/sdb3
Running: partclone.dd -z 10485760 -N -L /var/log/partclone.log -s /dev/sdc3 -O /dev/sdb3
Running: partclone.dd -z 10485760 -N -L /var/log/partclone.log -s /dev/sdc3 -O /dev/sdb3
Running: partclone.dd -z 10485760 -N -L /var/log/partclone.log -s /dev/sdc3 -O /dev/sdb3
Running: partclone.dd -z 10485760 -N -L /var/log/partclone.log -s /dev/sdc3 -O /dev/sdb3
Running: partclone.dd -z 10485760 -N -L /var/log/partclone.log -s /dev/sdc3 -O /dev/sdb3
Running: partclone.dd -z 10485760 -N -L /var/log/partclone.log -s /dev/sdc3 -O /dev/sdb3
~~~

Steven

[Steven Shiau]

BTW, please also show the green command you have there, like this one:
https://clonezilla.org/clonezilla-live/doc/03_Disk_to_disk_clone/images/ocs-08-3-command-to-run.png

Steven

[Steven Shiau]

OK, I believe I found the way to reproduce this issue. We will try to fix asap.

Steven

[ccedricc]

Thanks, i hope this image will be useful to fix this issue.

It has been uploaded on a free Dropbox account, normally only valid during 1 month.
If you need it, this is system passwords :

• the root password is "azer"
• an user "test" with password "azer"

I split the original file "8G_clone_loop.img" in 8 parts with this command line :
split -b 1000M 8G_clone_loop.img 8G_clone_loop.img-split-

To get the original normally this one :
cat 8G_clone_loop.img-split-* > 8G_clone_loop.img

8G_clone_loop.img-split-aa (www.dropbox.com)
8G_clone_loop.img-split-ab (www.dropbox.com)
8G_clone_loop.img-split-ac (www.dropbox.com)
8G_clone_loop.img-split-ad (www.dropbox.com)
8G_clone_loop.img-split-ae (www.dropbox.com)
8G_clone_loop.img-split-af (www.dropbox.com)
8G_clone_loop.img-split-ag (www.dropbox.com)
8G_clone_loop.img-split-ah (www.dropbox.com)

Below the green command when with the same choice :
/usr/sbin/ocs-onthefly -g auto -e1 auto -e2 -r -j2 -sfsck -k0 -p choose -f sdc -d sdb

[Steven Shiau]

Thanks for your feedback.
Please give testing Clonezilla live >= 3.1.1-12 or 20230807-* a try:
https://clonezilla.org/downloads.php
This issue should have been fixed.
Please let us know the results. Thanks.

Steven

[ccedricc]

Thanks you.

No loop for LVM partition with Clonezilla live 3.1.1-12.
The USB key has been well cloned.

Just a lot of alert : "warning could not determine physical sector size" (Just one line in log file)

A one moment "Clonezilla" ask me to type ok (OK: ok)

Here logs files and and a "fdisk -l".

Thanks again.

[Steven Shiau]

I believe you can ignore the warning: Could not determine physical sector size for /dev/sdb.
It's just a warning, so harmless.

Steven

[Declan McArdle]

Hi, I have/had the same issue with the looping problem.

It appears to have been fixed in version 3.1.1-12.

Thanks.

[Steven Shiau]

Great. Thanks for confirming that.

Steven

[goose]

For those still experiencing an issue with this, I deleted my reddit account and all its content in protest of the API changes, so my solution/workaround is no longer posted there. I'll repost it(and slightly re-word it for clarity) here, and hopefully it will be helpful to someone else in the future.

---

I imagine it is related to LVM. Here's what I did:

For reference, /dev/sda was my original SATA SSD that I was cloning from, /dev/nvme3n1 was the new NVMe SSD I was cloning to.

My /dev/sda had 3 partitions:

/dev/sda1 - /boot/efi (~536 MB)
/dev/sda2 - /boot (~496 MB)
/dev/sda3 - LVM (~476 GB)

Both drives were listed as 512 GB, but sometimes that "round number" has a difference of bytes. I was worried my new disk may be just a few bytes smaller than the old disk I was transferring from. So I shrunk my LVM PV partition (/dev/sda3) down to 460 GB to ensure it would fit on my new drive.

1. Resize the PV:
   pvresize --setphysicalvolumesize 460G /dev/sda3
2. Back up the existing partition table:
   sfdisk -d /dev/sda > /root/partition_table.dump
3. Initialize fdisk on the disk (Not the partition):
   fdisk /dev/sda
4. Use the p command to view the current partition table and verify the "start" and "end" values are displayed as sectors. Make a note of the "start" value.
5. Use the d command to delete the current partition entry for sda3.
6. Use the n command to recreate the primary partition entry 1 for sda3 in desired size, taking care to specify the location of the first sector exactly as it was (i.e. the same number as was displayed in the "start" value of the p command). If asked if you want to overwrite/delete the LVM signature, say "No".
7. Use the t command to set the type of the partition 1 to "Linux LVM".
8. Use the w command to write the updated partition table and exit fdisk.

If the partition resize is successful, you can now run pvresize /dev/sda3. If you encounter an error, you can use sfdisk /dev/sda < /root/partition_table.dump to restore the partition table to what it initially was. The steps to safely resize the PV were taken from here.

Once that was done, I fired up CloneZilla and got stuck in the clone loop. So I dropped into a shell and cloned it manually.

1. Ensure there were no partitions on my new drive: wipefs -a /dev/nvme3n1
2. Get the partitioning of the old drive:
   sfdisk -d /dev/sda > /tmp/partitions.txt
3. Load the exact same partition alignments into the new drive:
   sfdisk /dev/nvme3n1 < /tmp/partitions.txt
4. Manually clone each partition:
   dd if=/dev/sda1 of=/dev/nvme3n1p1 bs=4M status=progress
dd if=/dev/sda2 of=/dev/nvme3n1p2 bs=4M status=progress
dd if=/dev/sda3 of=/dev/nvme3n1p3 bs=4M status=progress
5. Wait. While /dev/sda1 and /dev/sda2 copied in seconds, /dev/sda3 took about 4 hours.
6. Once it's done, re-install Grub. First, make a directory to mount everything to so you can chroot:
   mkdir -p /mnt/newdrive/boot
7. Mount the new /boot partition
   mount /dev/nvme3n1p2 /mnt/newdrive/boot
8. Mount the new /boot/efi partition
   mount /dev/nvme3n1p1 /mnt/newdrive/boot/efi
9. Mount the necessary virtual filesystems
   for i in /dev /dev/pts /proc /sys /sys/firmware/efi/efivars /run; do mount -B "${i}" "/mnt/newdrive${i}"; done
10. chroot into your new drive:
    chroot /mnt/newdrive
11. Reinstall GRUB
    grub-install /dev/nvme3n1
12. Update the GRUB config
    update-grub

In terms of getting my OS transferred over, that's all it took. Shutdown, pulled the CloneZilla drive, pulled the old SATA drive, booted into BIOS and set my boot preference to the NVMe drive, and voila.

Once booted into the new drive, I re-expanded my LVM partition. You do this in reverse order of how you shrink it. So to shrink it, you do [pvresize] => [fdisk]. To grow the LVM PV, you do [fdisk] => [pvresize].

(Side note, my NVMe drive became /dev/nvme0n1 after rebooting)

1. Expanding my /dev/nvme0n1p3 partition to 100% using fdisk:
   This is essentially the same steps for shrinking the PV that were listed above, but in reverse. In a nutshell, run fdisk on /dev/nvme0n1, note the start sector of partition 3, delete partition 3, create a new partition 3, use the same start sector, but this time set the end sector to the last available.
2. Notifying the OS of the partition changes:
   Usually Linux only checks the partitions when it starts up, so it won't notice if you've changed them without a reboot. Instead of rebooting, you can ask the OS to re-check the partitions with partprobe /dev/nvme0n1(or partx -u /dev/nvme0n1 if your OS doesn't have partprobe as mine didn't, they do the same thing).
3. Grow the LVM PV:
   Run pvresize /dev/nvme0n1 to resize the LVM PV to 100% of the available partition space

And that's it. Mostly. I had some further issues to work through, but that was related to my motherboard not wanting to boot from an NVMe drive, not anything related to cloning the OS.