Error backing up NTSF parition

[Anonymous]

Don't know how to solve this error;

"Unable to fully process the image associated with the following partitions:
sda2: ntfs
This can happen when loading images which Clonezilla was unable to completely backup. Any other filesystems within the image should be restorable as normal."

Never used Clonezilla previously.

[Rescuezilla]

Hi Anonymous,

This error message is saying that the second partition (sda2) failed to backup, but any other partitions you within that image did backup correctly. I should change the error message displayed after scanning the image to say 'Rescuezilla or Clonezilla' such issues can happen with Rescuezilla and not just Clonezilla.

It's very likely you had an error message during your original backup operation. It would have shown an error message dialog, and the summary page would have also repeated a partition failed to backup.

The root cause is almost certainly your source NTFS filesystem was 'hibernated'. One easy way to test this idea is to try selecting the 'Restart' item from the Windows shutdown menu, rather than 'Shutdown'. Then immediately boot into the Rescuezilla USB stick and make a backup. It's possible to disable hibernate entirely from Windows, but that's obviously not ideal.

After making backup on a disk containing an NTFS filesystem that's not hibernated, the backup will succeed and subsequent scanning of that new backup will reveal no issues.

[Anonymous]

Hey thanks for the quick response! Did as you instructed, fresh Windows boot, inserted Rescuezilla USB, clicked Restart, inserted USB external hard drive, clicked Backup, but now getting this;

Mounting volume... FAILED
Attempting to correct errors...
Processing $MFT and $MFTMirr...
Reading $MFT... OK
Reading $MFTMirr... OK
Comparing $MFTMirr to $MFT... OK
Processing of $MFT and $MFTMirr completed successfully.
Setting required flags on partition... OK
Going to empty the journal ($LogFile)... OK

Windows is hibernated, refused to mount.
Windows is hibernated, refused to mount.
Remount failed: Operation not permitted

[Rescuezilla]

Hi Anonymous,

This error message indicates your Windows NTFS partition continues to be hibernated.

When you restarted your computer did you boot directly into Rescuezilla? If you allowed Windows to boot back up again then shutdown will have hibernated the disk again. Selecting restart from the Windows start menu and booting directly into Rescuezilla should be enough to ensure the disk isn't hibernated.

I don't think you have any other Windows installations on the disk your making a backup of, so I think it's worth trying one more time to restart into Rescuezilla and make another backup. If it fails again, you can try the answer from the AskUbuntu question I linked to by opening a terminal from the Desktop and running the following:

sudo ntfsfix /dev/sda2

The ntfsfix command above should clear the hibernated state of the Windows drive. But it's a little bit dangerous any unsaved work it may be lost. Also will make the disk will do a (fast but scary looking) consistency check next time it boots into Windows. But it will definitely allow Rescuezilla to make a backup. I suggest it's worth trying again to restart directly into Rescuezilla before using the ntfsfix command.

[Anonymous]

Yes, thanks, that's why I inserted the Rescuezilla USB before I click Restart in Windows so it directly boots Rescuzilla immediately afterwards. I'll go try the 'ntfsfix' command method next.

[Anonymous]

So this command gives this error;

bill@Toshiba-Mint20:~$ sudo ntfsfix /dev/sda2
[sudo] password for bill:
Mounting volume... Windows is hibernated, refused to mount.
FAILED
Attempting to correct errors...
Processing $MFT and $MFTMirr...
Reading $MFT... OK
Reading $MFTMirr... OK
Comparing $MFTMirr to $MFT... OK
Processing of $MFT and $MFTMirr completed successfully.
Setting required flags on partition... OK
Going to empty the journal ($LogFile)... OK
Windows is hibernated, refused to mount.
Remount failed: Operation not permitted
bill@Toshiba-Mint20:~$

[Christopher Weis]

If this is a windows 10 machine, have you tried shutting down the computer
while holding the left shift key to ensure complete shutdown?

(Edit by Rescuezilla: Removed Sourceforge's automatic email-style quotation of entire previous comment from the Anonymous user)

[Anonymous]

Yes, I did that too. As I explained elsewhere, I found I had to explicitly toggle ntfs hibernation Off before the partition mount and backup works.

[Anonymous]

Then I went into Windows and turned off hibernation through 'powercfg /h off'. I get the following error;

/mnt/backup/2021-08-08-1959-img-rescuezilla/parts: Unable to fully process the image associated with the following partitions:
sda2: ntfs
This can happen when loading images which Clonezilla was unable to completely backup. Any other filesystems within the image should be restorable as normal.
'

[Rescuezilla]

The most recent error message indicates the image you created at 2021-08-08-1959 had the hibernated partition problem. But that is referring to one of the images you just created earlier, right?

Now that hibernation is off, if you reboot and create a full backup the image should work without issues.

Any errors that appear during the "image scanning" process should only refer to images you created earlier. Check the name of the image to determine when it was created. Is this the case? By the way, you can delete the old images if you'd like (after very carefully checking the timestamps and ensuring you are only deleting unwanted images).

[Anonymous]

Yes, you are right. I did not notice the timestamp referenced a previously failed backup due the ntfs hibernation issue.

After I explicitly issued 'powercfg.exe /h off' in Windows, and rebooted I succeeded with 'sudo ntfsfix /dev/sda2';

bill@Toshiba-Mint20:~$ sudo ntfsfix /dev/sda2
[sudo] password for bill:
Mounting volume... OK
Processing of $MFT and $MFTMirr completed successfully.
Checking the alternate boot sector... OK
NTFS volume version is 3.1.
NTFS partition /dev/sda2 was processed successfully.
bill@Toshiba-Mint20:~$

Afterwards, I also succeeded completing the Backup!!

I also did scenario testing with toggling ntfs hibernation On/Off, and found that you have to explicitly toggle ntfs Off. If ntfs is toggled On, these error messages appear;

'sudo ntfsfix /dev/sda2'

Failed to run command: ntfsfix --clear-dirty /dev/sda2

Mounting volume... FAILED
Attempting to correct errors...
Processing $MFT and $MFTMirr...
Reading $MFT... OK
Reading $MFTMirr... OK
Comparing $MFTMirr to $MFT... OK
Processing of $MFT and $MFTMirr completed successfully.
Setting required flags on partition... OK
Going to empty the journal ($LogFile)... OK

Windows is hibernated, refused to mount.
Windows is hibernated, refused to mount.
Remount failed: Operation not permitted.

'Rescuezilla Backup'

Failed to backup partition /dev/sda2

The command used internally was:

partclone.ntfs --logfile /var/log/partclone.log --clone --source /dev/sda2 --output - | pigz --stdout -6 | split --suffix-length=2 --bytes=4GB - /mnt/backup/Samsung/2021-08-09-1941-img-rescuezilla/sda2.ntfs-ptcl-img.gz.

The output of the command was: Partclone v0.3.13 http://partclone.org
Starting to clone device (/dev/sda2) to image (-)
Reading Super Block
ntfsclone-ng.c: NTFS Volume '/dev/sda2' is scheduled for a check or it was shutdown
uncleanly. Please boot Windows or fix it by fsck.
Partclone fail, please check /var/log/partclone.log !

Thanks for all your detailed help!! I'm proud to be a monthly support contributor through Patreon!

Btw, I suggest a minor enhancement ... when the target drive has insufficient capacity for the backup, there should be a warning or resulting message stating insufficient disk capacity. Currently it just hangs there with no further progress.

[Rescuezilla]

Oh, that's very interesting that hibernation had to be switched off. I really would have thought a Windows 'Restart', or as Christopher Weis said above, a Windows 'Shift + Shutdown' would have been enough to disabled hibernation, but apparently not.

Now that hibernation is switched off, manually running ntfsfix should no longer ever be required.

Btw, I suggest a minor enhancement ... when the target drive has insufficient capacity for the backup, there should be a warning or resulting message stating insufficient disk capacity. Currently it just hangs there with no further progress.

Thanks for the bug report around the insufficient disk capacity issue. I agree it's definitely something that should have a warning message associated with it. I have created task #253 to capture the task, and will also try and complete task #182 to try and improve graphical display of free/used space.

Thanks for all your detailed help!! I'm proud to be a monthly support contributor through Patreon!

No problem! I didn't realize you were a Patreon supporter until just now! Thanks for the support! :)

I hope you and anybody else reading gives Rescuezilla a 'like' on AlternativeTo (if you haven't already) so that more people can discover the program! :)

[Anonymous]

Yes, I just went through re-testing both your suggestion to Windows 'Restart' directly into Rescuezilla , and Christopher Weis' to do a Windows 'Shift + Shutdown'. Neither of them work to disable hibernation. Had to explicitly issue Windows 'powercfg.exe /h off' After that it works!

So will you consider building into a future release to automatically disable/re-enable hibernation on Windows ntfs partitions in the backup process?

[Rescuezilla]

So will you consider building into a future release to automatically disable/re-enable hibernation on Windows ntfs partitions in the backup process?

It's not possible for Rescuezilla to modify Windows' hibernation using powercfg.exe /h off, and usually this setting shouldn't need to be changed because a restart is typically sufficient.

But you're right, I need to improve Rescuezilla's user-interface around reporting hibernated partitions to users, and providing an easy way to automatically fix the issue (such as a checkbox to clear the hibernation file with a warning of the dangers of doing this). Since ntfsfix didn't work for you, there is a dangerous remove_hiberfile option for mount.ntfs seems to make the most sense.

I have created GitHub task #254 to capture this task.

[Anonymous]

Thanks! Right, I didn't mean to imply that the 'powercfg' Windows command can be executed through Rescuezilla.

[Anonymous]

Just curious...I never encountered this ntfs hibernation issue with previous versions of Rescuezilla, and I've used it across 7 different computers, a mix of pure Windows, dual boot Linux/Win, and Mac/Win. What changed?

[Rescuezilla]

What's changed is Rescuezilla v2.2 completed the implementation of the remaining Clonezilla image restore logic to improve handling of many corner cases (task #146). As per the CHANGELOG:

* Implemented remaining Clonezilla image restore logic to improve handling of many corner cases: (task #146)
  * Grows filesystem to fit partition size (but almost always filesystem size is already equal to partition size)
  * Runs filesystem check on all restored filesystems
  * Clears the NTFS volume dirty flag, for source during backup/clone and destination for restore/clone
  * Removes the udev MAC address records (if any) using Clonezilla script
  * Re-install the syslinux bootloader (if any) using Clonezilla script
  * Re-install the GRUB bootloader (if any) using Clonezilla script
  * Update initramfs (if any) using Clonezilla script
  * Relocates all NTFS filesystems using geometry from sfdisk or (if available) EDD (Enhanced Disk Device)
  * Restores LVM VG metadata with --force to match a recent Clonezilla patch referring to thin-pool LVs 
  * Ensures each LVM VG metadata once is only restored once, to support newer versions of vgcfgrestore
  * Updates EFI NVRAM for the boot device

This includes using ntfsfix just like Clonezilla (to clear the volume dirty flag), which is where the hibernation issue seems to be currently tripping up. Rescuezilla v2.2 needs to be able to mount, modify and resize partitions to handle many more corner cases that have been implemented in Clonezilla for years/decades that were finally handled with Rescuezilla v2.2 (like NTFS relocation, growing the filesystem etc).

It does make sense that partclone itself might have been able to make a backup of a hibernated drive (it can be read as read-only), so previous versions of Rescuezilla may not have exhibited this issue. I actually haven't experimented with that very closely: I have always tested with a mix of Clonezilla and Rescuezilla images, so it's not clear to me when this problem first occurred in Rescuezilla. It's definitely happens to me when using Clonezilla even years ago.

I will figure out if it's possible to run a backup at degraded performance when doing a backup of a hibernated disk, and possibly warn the user and simply make the backup without the handling of advanced Clonezilla corner cases during restore operations.

[Anonymous]

I too have the same issue when cloning - with hibernation 'on' even when I KNOW that I have shut down windows correctly..... I will revert to an old version of RZ until fixed!

[Rescuezilla]

Thanks for letting me know.

Rescuezilla v2.3 will be released in November 2021 (less than 3 months from now). As mentioned, task #254 will re-evaluate this NTFS hibernation issue.

Until then, Rescuezilla v2.1.3 should be sufficient for you for backup/restore. Though v2.1.3 doesn't include the direct device-to-device 'Clone' feature which it sounds like you may be trying to use (with the workaround before the feature was added was to make a backup to a temporary drive, then do a restore to the new drive).

[Anonymous]

Thanks - I went back through older versions and ended up using a version 1 to manage an image with hibernation still enabled (though of course correctly shut down pre imaging) (sorry - I meant image not clone above). Thanks again

[Rescuezilla]

Thanks for the information. I would be VERY hesitant to use such an old version. If you insist, then only use v1.0.6.1 of the v1.y.z series. Even then I still would recommend against it unless you very carefully read the bug advisory section of that release on its release page.

I know that removing Rescuezilla v2.2's expansion of ntfsfix and putting back partclone's --force would fix these issues a small number of people are having. But there are tradeoffs involved and they require a lot of testing.

I'll keep investigating.

[Anonymous]

Yes - even 1.0.6.1 produced unpredictable results..... So I have opted to disable hibernation before shutting down which of course deletes hiberfile.sys. RZ then flawlessly images the disk and re-deploys perfectly too :)

[Anonymous]

Yes, that is my temporary workaround too. And can also confirm that RZ worked great after that, both backup and a partition restore.

[Anonymous]

Btw, I saw these error messages right after boot on an iMac (Intel-based);

Couldn't do proper screenshot so just took a pic of the screen

[Rescuezilla]

This iMac query ideally should have been a separate forum topic.

The error message you see on your Intel-based iMac refers to MokListRT, which is a part of the EFI Secure Boot process.

It's not clear when the error is occurring? Is it only when you're trying to boot Rescuezilla? Which version are you trying to boot (Hirsute or Focal)? What operations have you done so far. Just trying to boot Rescuezilla?

I can confirm I have successfully run Rescuezilla on a single Intel-based APFS Mac Mini several times and was able to do test backup/zero disk/restore without any issues or configuration changes. For completeness, I note Rescuezilla is not yet APFS filesystem aware (task #65), so the 50GB of files on a 1TB drive provides a 1TB image, not a 50GB image. But this fact means there is little that can ever go wrong.

I believe Clonezilla may already do APFS in a filesystem-aware way.