Using latest stable (or testing) clonezilla causes very slow disk access on...
A partition and disk imaging/cloning program
Brought to you by:
steven_shiau
Menu
▾
▴
Not sure why, but when using the latest clonezilla 2.4.7-8 i686-pae or 2.4.7-23 i686-pae + xz compression (no encryption), the cloning becomes incredibly slow after several minutes (about halfway into the clone).
Before this happens, the read speed is great and CPU is fully utilized. After it happens, CPU cores are not utilized at all - occassionally one or two cores start "processing" some data but the read speed comes to a crawl. Several 1-5 megabytes (maybe less, I lost my patience :p ) instead of hundreds of megabytes per second. When I open top, I can see that wait time goes to about 10-20% even though disks are clearly not hitting their transfer limits.
If I then cancel the cloning and restart it again, it's slow from the first second, so for example cloning the first 500MB partition takes like 5 minutes when it should be 5 seconds.
Doing the same on the yakkety version (amd64 - 20160726-yakkety), this no longer happens.
I suspect it may have something to do with:
a) my platform - Z170, "fairly" new
b) the source disk - an NVMe SSD (128GB with about 25GB used)
c) amount of RAM in the system (not likely, but I have 64GB).
Likely A or B are fixed by using a newer kernel of yakkety or something else is happening.
If you need more info or logs, let me know.
Thanks for sharing that. Since Clonezilla live 20160726-yakkety works, I think maybe it's due to the linux kernels hardware supporting related. Another updating in the Linux kernel for Clonezilla live might fix this issue. Let's wait the upstream Debian to have newer Linux kernel in Sid and the new generated Clonezilla live will use that.
Steven
So I tried with the latest kernel and the situation was the same ( 2.4.7-26 i686-pae) . So I did many tests and such and at first I thought it could be TCP with its window scaling, but even physical disks exhibited the exact same behavior when used as /home/partimag . I.e. unaffected (=fast) reads and super slow writes (I'm talking 150MB/s reads and 1-2MB/s writes).
I was playing around with things and googling stuff and eventually I found THIS:
http://stackoverflow.com/questions/30519417/why-linux-disables-disk-write-buffer-when-system-ram-is-greater-than-8gb
Sure enough. Checking the values on my system:
nr_dirty 0
nr_writeback 0
nr_writeback_temp 0
nr_dirty_threshold 0
nr_dirty_background_threshold 0
Echoing 1 into the parameter ( echo 1 > /proc/sys/vm/highmem_is_dirtyable ) caused this:
nr_dirty 0
nr_writeback 0
nr_writeback_temp 0
nr_dirty_threshold 2704106
nr_dirty_background_threshold 1352053
And all of the transfer speeds immediately shot back up.
This also explains why yakkety worked - because it was the amd64 version which doesn't have this issue (as it only affects x86 kernels).
I use the i686-pae version because it's universal for all the devices - 32/64bit, uefi/bios, all of it works from the same version.
So in the end it was actually the c) option that I said was not very likely in my original post :D
With this in mind (and how it impacts backups), could clonezilla make it so that this parameter is auto-set by default? It doesn't seem to have any negative impact otherwise.
For your suggestion is for x86 (not x86-64/amd64) arch, if the RAM size is larger than 8 GB, we should run
echo 1 > /proc/sys/vm/highmem_is_dirtyable
?
Thanks.
Steven
Yup! I tried a few runs with/without this parameter and pretty much unusable clonezilla turned into completely normally operating clonezilla for me on my system after I echoed 1 into /proc/sys/vm/highmem_is_dirtyable .
Only for the x86 (i686-pae) version, of course. The x86-64/amd64 doesn't need this and works properly out of the box.
OK, we will think about it. For the moment, you can have your own customized version by editing the file syslinux/isolinux.cfg or syslinux/syslinux.cfg, put
ocs_prerun="echo 1 > /proc/sys/vm/highmem_is_dirtyable"
If you want to remaster iso file, check this:
http://drbl.org/faq/fine-print.php?path=./2_System/87_create_clonezilla_iso_from_zip.faq#87_create_clonezilla_iso_from_zip.faq
Steven
Will this affect uefi-boot? In both of the CFGs, I can see the "this is the boot menu for BIOS machine" - and the machine that I'm booting on is a uefi-boot machine.
No, I do not think it will affect uEFI boot. It's something after booting while enter GNU/Linux environment.
Steven
BTW, your request is now in the git repository. The next testing Clonezilla live will use that. Thanks for suggestion.
Steven
Awesome! I see it - 54f4af8353f1705752d140f8e0689ce7b1a8727c . I'll try it out immediately when it's available~ :) Thank you very much.
Cloud you please give Clonezilla live 2.4.7-30 or 20160830-* a try?
http://clonezilla.org/downloads.php
This issue should have been fixed. When the Linux kernel is x86, not x86-64, and the RAM size is larger than 8GB, /proc/sys/vm/highmem_is_dirtyable will be set as 1.
Please let me know the results.
Thanks.
Steven
I tested it and there is some kind of issue now. After selecting uefi boot and the regular option as always, I get into some kind of loop during the init stage of the system.
This message keeps repeating over and over:
mount: mounting /dev/sdc1 on /live/medium failed: invalid argument.
Eventually, the system stops trying and shows the ash of busybox, stating:
(initramfs): unable to find a medium containing a live filesystem
I tested sha512sum of my iso and it's correct.
Last edit: Little Vulpix 2016-08-30
Not sure if this is related to the new Linux kernel (4.7) used in this testing release. Therefore, could you please give clonezilla live 20160830-* a try? They use older Linux kernel so the results might be different.
BTW, 2 more questions:
Steven
Hello Steven! I'll test the other version of clonezilla.
I think /dev/sdc1 is the usb disk, but I have no way of knowing since I don't really get into any sensible type of environment. Can I figure this out somehow? If it is the USB stick, then the filesystem is fat32.
And yes, I'm testing UEFI + i686-pae version (like all those times before). The machine where I'm testing it has UEFI-CSM / BIOS boot disabled.
I'll post here in the evening with results of my test. I also tried with the previous version of clonezilla beta to make sure my USB stick is created properly (I use rufus), and it worked just fine.
Last edit: Little Vulpix 2016-08-31
That's weird... Why the same method that your USB flash drive can not boot. How about trying to format your USB flash drive again, and use the method shown here
http://clonezilla.org/liveusb.php
to install it again?
BTW, for UEF boot, the right version of Clonezilla live you should use is AMD64, not i686 version. Unless your CPU only support x86, no supporting for x86-64.Otherwise there will have some problem to update the boot manage in your BIOS.
Steven
Hi!
I feel like somewhere along the lines my initial issue got forgotten.
Basically, the reason I use the i686-pae version is because it's universal. It works on 1gb x64 tablets with x86 uEFI, it works on an old pentium 2 system, it works on my skylake i5 as well. I know I should use the 64bit version - and that version doesn't need the "highmem is dirtyable" parameter at all, but I'd constantly need to recreate my usb stick when I'm performing various backups. This way, it's just fine. No issues whatsoever.
Additionally, I tried re-creating the flash 3x (using rufus -> http://rufus.akeo.ie/ ) and it never worked for the debian ones, but worked for the previous debian version (29) and also worked for yakkety and xenial (both of them). Tested just now
Hi,
What I meant is, surely you can use i686-pae to boot a uEFI machine. However, IIRC I encountered some issues about updating EFI boot entry in NVRAM. Maybe now things have been imporved since I have never tried to boot x86-64 uEFI machine with i686 version of Clonezilla live anymore. If you are sure it works there, please let us know.
As for the issue you mentioned about Clonezilla live 2.4.7-30 on USB flash drive with FAT file system, I confirmed that I can reproduce this issue. There is no such issue to boot Clonezilla live ISO (hence CD). Therefore this issue is specific to Linux kernel 4.7 with FAT in this command:
mount -t vfat -o ro,noatime /dev/sdc1 /live/medium
mount: mounting /dev/sdc1 on /live/medium failed: Invalid argument
I will dig more.
Steven
Apparently the mount command comes with busybox fails to mount the FAT file system from Linux 4.7. I will need more time to figure it out. For the time bing, there is a workaround for you. Format your USB flash drive as NTFS file system. The put Clonezilla live on it and make it bootable. This should avoid the issue.
Steven
OK, I have fixed this issue by patching live-boot to include nls_ascii module in the initramfs. Please give Clonezilla live 2.4.7-31 a try:
http://clonezilla.org/downloads.php
Let us know the results. Thanks.
Steven
Hello Steven!
Thank you so much for your hard work on this issue. I confirm it works with 2.4.7-31 on all of my devices without an issue, and on the 64GB RAM one, it properly enables highmem_is_dirtyable so the clone is incredibly fast (~6.5GB/s with parallel gzip!)
Thank you once again. You can mark this issue as resolved!