#365 Sees HDD for read / write but not for imaging

[Paul]

clonezilla-live-2.7.2-38-amd64
Setup: 1 x SSD W10 (sda) - 1 x HDD W10 (sdb) - 1 x Storage HDD (sdc) on Lenovo S20 PC. Both W10 drives are bootable.

Clonezilla will see all HDD's and partitions for selecting the read write disk but when you get to select the HDD to image it does not see the 1 x HDD W10 above 'sdb'. Tried multiple times and no luck. both W10 disks are bootable and fully functional.

I then used and old CD live copy of clonezilla I have lying around 'Clonezilla r2.1.2 - 20-amd-64 and it worked fine seeing all disks, sda, sdb and sdc.

[Steven Shiau]

Thanks for your feedback. Please boot Clonezilla live 2.7.2-38, enter command line prompt, then run:

1. sudo -i
2. cat /proc/partitions
3. blkid

Please post the results of 2 & 3.
Thanks.

Steven

[Paul]

I don't know what information I'm posting so I redacted some ID's.
Both files attached > cat /proc/partitions and blkid

Note: I maybe the only one on the planet to have ever or will ever hit this bug. I'm not to concerned for I will be removing the missing HDD soon as I can also use an older version of CZ to image it.

[Steven Shiau]

This is interesting... It looks normal to me...
Is this issue reproducible on testing Clonezilla live 20210609-impish?
https://clonezilla.org/downloads.php

Steven

[Paul]

No good, bug still there. See the screen shot for I also tried another older version and bug did not exist.

[Steven Shiau]

I tried to have 3 disks in the virtual machine to reproduce this issue. Somehow I just can not reproduce it...
Is that possible I can remote login to that machine to diagnose it? Say you boot Clonezilla live on the machine, configure the network and start ssh service, and I use ssh to login that.
If so, please email me at steven@clonezilla.org
Thanks.

Steven

[Paul]

Sorry Steven but that's not going to happen due to:
1- My internet is a pocket USB wifi so connecting CZ would be a time consuming nightmare if at all possible, drivers, dependencies.
2-Long story short, no, sorry.

Anyway, as mentioned previously I'm probably the only one to have ever or will ever hit this bug so I'm not to fussed as I will be removing the HDD soon. The only thing I can think of is that the disappearing disk also has W10 installed on it. Maybe CZ can only recognise one disk with W10 for imaging. Have you tried having two disks with W10 installed on them.

[Steven Shiau]

Sure, I understand that.
"Maybe CZ can only recognise one disk with W10 for imaging. Have you tried having two disks with W10 installed on them. " -> No, Clonezilla will detect all disks, no matter it's Win10 system, data disk, GNU/Linux...
Actually what I have tried was very similar to what you have:

root@debian:~# blkid | sort
/dev/loop0: TYPE="squashfs"
/dev/sda1: BLOCK_SIZE="512" UUID="D27412037411EB4D" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="0ec017af-bc82-4598-bb38-839bb8c44527"
/dev/sda2: UUID="6A8D-DD1F" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="a9427f00-e1d1-4bc5-9cd0-e23833e6fe62"
/dev/sda3: PARTLABEL="Microsoft reserved partition" PARTUUID="699de00f-a59f-40d1-8a0d-89e76e0319d4"
/dev/sda4: BLOCK_SIZE="512" UUID="709C8E839C8E4396" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="1b16b769-a88a-475f-b767-d829f18ba746"
/dev/sdb1: BLOCK_SIZE="512" UUID="75BF35CA12315E98" TYPE="ntfs" PARTUUID="4b230920-2962-8749-9c43-c4497b48ebca"
/dev/sdc1: LABEL="SYSTEM" UUID="EAD8-EB36" BLOCK_SIZE="512" TYPE="vfat" PARTLABEL="EFI system partition" PARTUUID="99290d0b-553c-4139-8ae3-09971aabae8c"
/dev/sdc2: PARTLABEL="Microsoft reserved partition" PARTUUID="63391d78-1fa9-41eb-8e99-3bc57751285a"
/dev/sdc3: LABEL="Windows" BLOCK_SIZE="512" UUID="F0BEE71EBEE6DC5A" TYPE="ntfs" PARTLABEL="Basic data partition" PARTUUID="59e6fb44-a0de-4867-acfa-54a542854120"
/dev/sdc4: BLOCK_SIZE="512" UUID="EEB8004FB800192B" TYPE="ntfs" PARTUUID="cf12cd4c-4fec-4f8c-93e1-7cd393532f35"
/dev/sr0: BLOCK_SIZE="2048" UUID="2021-06-09-14-01-12-00" LABEL="2.7.2-39-amd64" TYPE="iso9660" PTUUID="6dd306b8" PTTYPE="dos"

And Clonezilla live 2.7.2-39 can detect all of them, as attached.

Steven

[Paul]

I noticed you used *.2-39 so I tried it but still the same.

Give me a few days and If I get the time what I will do is format the HDD test to see if it is picked up for imaging and if so I will then clone back an old Copy of the same W10 and see what happens.

[Paul]

Steven

Ok I did a restore, see the attached image.

It seems that it's the way that W10 sits on that HDD. As said don't worry about it.

[Steven Shiau]

Thanks. I am wondering what kind of Win10 on that? Encrypted? Or anything special? What I have is a Win10 with GPT partition table and no encryption.

Steven

[Paul]

See the image attached for comparison to SSD, also note, no encryption or boot loader such as Grub used. To boot into the disappearing disk I use F12 on the Lenovo which brings up the boot menu, other PC's is usually F8.

[Steven Shiau]

OK, thanks. So are you familiar with screen or tmux? If so, please enter it, then run:
sudo bash -x ocs-sr -x
Follow the normal procedure to save the image. When it shows the menu for selecting source disk, you can quit the problem. Then copy & paste the whole messages on the screen after you run "sudo bash -x ocs-sr -x". Post it. I can diagnose that.
Thanks.

Steven

[Paul]

My knowledge is rudimentary. You need to be more specific.

From what I understood of your instructions

Did sudo bash -x ocs-sr -x
Then CZ to select source
Then sudo bash -x ocs-sr -x

In all about 1000 lines of fast moving text but nothing for me to capture or know how to.

When it finished and I cancelled out of CZ back to the command line. Then what?
I looked at $ nano /var/log/clonezilla.log but it was empty.

You need to give me a command that will write the output to a file onto the USB drive sdd or something.

[Steven Shiau]

OK, here is what you can do:

1. Boot Clonezilla live 2.7.2-39 amd64
2. Choose to enter command line prompt
3. sudo -i
4. Modify the buffer lines of screen using this command:
   perl -pi -e "s/defscrollback 1024/defscrollback 1024000/g" /etc/screenrc
5. Enter screen by this command
   screen
6. Run this command:
   bash -x ocs-sr -x
7. Try to save an image in the normal way, when choosing the source disk, it should only show 2 disks, not 3, as you have mentioned.
8. Chooose to cancel selecting the source disk, and you will back to the command line prompt
9. Now press Ctrl-a then [, it will enter screen's copy mode
10. Type "?root@", then it should find "root@", press "n", then it will find the previous "root@" where you issue the command "bash -x ocs-sr".
11. Press "space" key to mark the 1st to copy, then press "G" to go to the last one, then press "space" key again. It will mark the last line. Now you have the all of the texts on the screen in the buffer.
    (You can refer to the manual of screen: http://web.mit.edu/gnu/doc/html/screen_11.html)
12. Use any text editor, say "vi" or "nano", i.e.,
    nano my-output.txt
    Then press Ctrl-a then ] to paste the texts from the buffer.
    Save it as a plain text file. Now you should have a file called my-output.txt, which contains the output I'd like to review. Post it and I will diagnose it.
    Thanks.

Steven

[Paul]

One thing noted was that CZ jumps to source. No read write disk selection, with that I noticed that in the output the disappearing HDD is there as 'sdc' but not in any form of ID such as type or number as the other two are further on the list.

[Steven Shiau]

Somehow your /dev/sdc contains ext4 file system info in the disk's metadata. You can run these commands in Clonezilla live command prompt:

1. sudo su -
2. ocs-get-dev-info /dev/sdc fs
3. blkid /dev/sdc
   From 2 & 3, it should report the file system is ext4. In that case, the whole disk (/dev/sdc) is treated as a block device with file system...
   Is that disk used for GNU/Linux before?

Steven

[Paul]

That would explain why it became visible on format and disappeared when I restored W10. In any case, why does this bug exist now. Previous versions of CZ never had a problem as I have also imaged ext4 - Riserfs and possibly others in the past. I guess I don't understand what you mean by treated as a block device.

[Steven Shiau]

Maybe this issue is due to a bug in blkid. It reports that, and it is not correct.
If you try older version of Clonezilla live, you should see that the blkid from it does not have this bug I believe.
If that's true, we should file a bug report to the upstream. However, here I am not able to reproduce this issue, so maybe you can help to report it.

Steven

[Paul]

> If you try older version of Clonezilla live, you should see that the blkid from it does not have this bug I believe.

I've done that multiple times? Sorry Steven but your last response made me think I've been wasting my time with some random guy from the internet. I had to go to CZ and check the Developers to see if you are actually part of CZ.

Whilst as said this bug will not affect me as I may also be the only one on the planet to ever hit or will ever hit this bug, it may affect others and other distros now or in the future in some other way, as such, You are welcomed to submit all I have submitted to the blkid maintainers or other.

I thought of encrypting the HDD whilst online to hopefully preserve the fS then zeroing it for extra data protection hence creating an image and uploading it, but would that make a 500GB image or a few MB image having been zeroed. In any case it will be very time consuming for something that may fail, as in, the FS may be destroyed by zeroing the HDD or hidden when encrypted.

The only other time I'm willing to spend on this is possibly in downloading for example the last 2 or 3 stable releases of CZ prior to 2.7.2-38-amd64 and you may find where the bug originates by package comparison.

[Steven Shiau]

"I've done that multiple times? Sorry Steven but your last response made me think I've been wasting my time with some random guy from the internet. I had to go to CZ and check the Developers to see if you are actually part of CZ." -> Haha, yes, but we did not identify the culprit is blkid. Hence I meant if you try to run "blkid /dev/sdc" and "ocs-get-dev-info /dev/sdc fs" using the older versions of Clonezilla live. Not to run the TUI only.
Thanks.

Steven

[Paul]

I think the bug exists with ocs scan on initial device scan. See the image ocs scan failure. The delay of about 20 seconds presents itself as a blank screen with a blinking curser as it also says 'Excluding linux raid member partition...' Is 2-7-2-38 seeing 'sdc' as a raid member disk when all older CZ versions do not?

[Steven Shiau]

In the old version of Clonezilla live, the command ocs-get-dev-info was ocs-get-part-info. Hence you have to use ocs-get-part-info instead of ocs-get-dev-info.
So even the old version of Clonezilla live, blkid report /dev/sdc as ext4?
Then maybe I am wrong. The culprit is not really on blkid.
For the moment, I have no idea why it, including ocs-scan-disk, can not be detected by Clonezilla live 2.7.2-38...

Steven

[Paul]

Steven it is being detected it's just being ignored. Go back and remember that for read / write sdc is available It's just not available for imaging. 2-7-2-38 "ext4.jpg" above in a previous post.

My theory. 2-7-2-38 sees the HDD as a raid so It ignores on scan for imaging but has no problem for mounting as home part image.

That disk has been like that for at least two years and has been imaged many times by CZ. Was it ever part of linux, yes. It was part of a multi disk fuse file system using Mergerfs years ago but again as previously said W10 fresh install over a year ago should have cleared all that.

Anyway. Like I said, it detects but it's being ignored for some reason now.

[Steven Shiau]

"My theory. 2-7-2-38 sees the HDD as a raid so It ignores on scan for imaging but has no problem for mounting as home part image. " -> It's very possible.
Maybe you can try to clean the RAID info from that disk. You can refer to this:
https://www.slac.stanford.edu/grp/cd/soft/unix/RAIDmetadata.html
//NOTE// Back up important data before you run that command. Just in case.

Steven