Hi, I was trying to make a backup of my hard-drive but the process failed an hour later at around 90% of the way. The error showed "Input/Output error", and after that another annoying box kept popping up without giving me the option to start over, report bug or even exit the application. I had to kill the process from the terminal. I'll attach screenshots of these errors.
I restarted the PC and made sure everything still worked fine. It did, but on the external drive (where I would store the backup from my pc) there was the folder that I named "backup" which I can't access nor delete, it says is corrupted or damaged... any help on how to get rid of it is much appreciated.
I tried again later after that with similar results. The backup process failed at around 20%, but this time the backup folder on the external drive can be accessed and shows a bunch of files and logs which I'll attach here.
For reference, the drive I'm trying to backup is 1TB (size shows 931.51GB) and contains 4 partitions. However only about half the space is taken by these partitions, and the rest remains unallocated. In the log ouput there are three drives: sda is PC I want to backup, sdb is the live rescuezilla usb and sdc is where I want to store the backup.
So what has happened here is part way through the Rescuezilla v1.0.6.1 backup process, the command pipeline that is conducting the backup has failed for some reason. Usually this is because the "partclone" utility that is internally being used is unable to understand the filesystem that is undergoing the backup process. Though for some users the root cause can vary: eg, instability in accessing the destination network share folder, running out of RAM or a variety of other reasons. The log your provided indicates the "partclone failed partially through the backup" but little else. As you have seen the unexpected exit of a component within the Rescuezilla command pipeline causes current versions of Rescuezilla to break in ways that the user is not expecting (task #29), leading to the output of another part of the pipeline being gobbled up and immediately displaying it, leading to several incredibly poorly worded error messages. This issue causes a clearly terrible user experience.
One bright side is the fact something went wrong is always extremely clear when this very rare issue occurs: no user will ever think that their partially completed backup that has spewed several error messages was a success. As part of the Rescuezilla v1.0.7, I am finally completely fixing the the brittle command pipeline that has affected all Rescuezilla (and before that all Redo Backup and Recovery) as part of Rescuezilla v1.0.7 scheduled to be released in mid-October 2020. I have listed the poor exit code handling and error message under the "bug advisories" section on the Rescuezilla release page, but until now I have been focusing on other Rescuezilla tasks. I'm sorry I haven't corrected this poor usability issue yet.
Now, as far as the issue around a folder named 'backup', that is indeed very strange. It may be related to permissions. Could you try launching Rescuezilla, launching the File Manager from the desktop and clicking your drive on the left-hand menu to mount your drive. Are you at least able to navigate this folder using the Rescuezilla's file manager?
You may be able to delete this folder from the file manager, but you may need to right-click "Open current directory in terminal" to carefully delete it as a 'root' user.
If you are able to access the folder named 'backup', please upload the rescuezilla.log file from your first attempt, it may shed some light to why the 'partclone' utility is failing. By the way, I am also very interested in seeing the files ending with "partX_partclone.log" because they may contain some information not found in the main Rescuezilla log file.
I am very happy to continue to provide assistance for as long as required. Thanks again for attempting to use Rescuezilla! Sorry for any inconvenience caused by my lack of having completed task #29!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I did as you suggested and opened the external drive from Rescuezilla and the files are all there, it seems is only the Windows machine that cannot read them. However, most of the files show in the description as "inode/x-corrupted type" (as opposed to "plain text document" for example) and cannot open any of them. From the terminal is clear that something is wrong with them. I'll attach a screenshot of this, and the "rescuezilla.log" that you requested.
On the second backup attempt all files are there and readable, I'll attach the log files requested. I've renamed them for convenience to tell which files belong to which backup attempt.
Thanks again for the help, and I'm really looking forward to that upcoming update!
The issue with your Toshiba 1TB destination drive is very worrying. It looks like it's not simply an issue related to permissions but an indication that your backup drive has some filesystem corruption. I recommend that you do not use the Linux file manager to modify that filesystem until the filesystem is repaired.
It's possible that the force kill of the Rescuezilla process you mentioned has left the filesystem in an inconsistent state. The best way to fix this is to boot Windows and run chkdsk to do a filesystem check of the NTFS partition of your Toshiba 1TB drive. Here's a chkdsk tutorial I found. You may wish to initially run the filesystem check without the "fix errors" flag, so it scans in read-only mode as a sanity check without modifying the filesystem in any way.
I examined the log files from your most recent reply and they contain no indications why the Rescuezilla command pipeline failed part way through the 'partclone.ntfs' command. Unfortunately Rescuezilla v1.0.6.1 does not log the exit codes from the Rescuezilla pipeline commands, nor the kernel diagnostic messages, which would both be very useful to debug this kind of issue.
I should mention also mention there is very real possibility that your Toshiba 1TB destination drive is developing issues for reasons unrelated to Rescuezilla. If your drive encounters eg, SATA write errors while making a backup, this could explain why the backup process froze and also why there is some corrupted files on the filesystem.
One way to test that hypothesis is to get another external harddrive unrelated to your Toshiba 1TB and try using Rescuezilla v1.0.6.1 to make a backup of your internal hard drive again. If this succeeds without issue it's an indication that it's your Toshiba 1TB has errors, rather than anything to do with Rescuezilla.
Another potentially helpful indicator is examine the kernel diagnostic messages for hard drive errors immediately after experiencing an error. To do this, open a terminal as the root user (again, by right-click "Open current directory in terminal") and type dmesg -Tw. If a harddrive is failing then certain kinds of access will add entries to the dmesg output (highlighted in bright red) saying something like eg, ATA read has failed to because a bad sector.
This issue may indicate a potential problem in Rescuezilla v1.0.6.1. I am very keen to get to the bottom of this issue with extremely high priority. Please be extra careful with your Toshiba 1TB drive until we have debugged exactly what is going on.
🎉
1
Last edit: Rescuezilla 2020-07-22
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I didn't even consider the hard drive as a point of failure, but I'm glad you brought that up. The hard drive is actually fairly new, bought just a few months ago, although it may be that I haven't used it long enough to noticed an issues until now. I did as suggested and ran chkdsk which discovered an issue with the drive. Unfortunately the command prompt wasn't very helpful as to what the issue was, so I simply added the /f flag to automatically fix it. A final check and no errors found, and all files were recovered and readable again.
Just in case I copied over all the files in the Toshiba 1TB drive and then I simply tried to run Rescuezilla again. It worked perfectly this time! So it seems that as you suspected there was an underlying issue with the hard drive for some reason. It's strange that it worked normally up until now, but hopefully it will continue to work normally.
As a little feedback on the use of Rescuezilla, it was not entirely clear when I could close the window. The little prompt at the end telling me how long it took made it fairly obvious, but still an option to "go back" or something more clear to know that the program is done. No biggy just my opinion (maybe a little too paranoid whether it worked or not at this point).
On recovering from the backup folder, I have a couple of questions. First of all, the hard drive had about half of unallocated space but at the prompt I could select which partitions I wanted to backup, and it didn't appear to me that this space was included. Currently the backed up folder sits at 130GB. Could this be a problem if I attempt to recover the system into a 500GB drive (which is about the space that is currently taking)? And also, are there any risks to the PC if I attempt to recover from the files taken?
For reference I'm attaching the files for the latest, successful backup in case there's something there that helps. Please let me know if you'd like to see something about the others.
Thank you once again for all your quick and kind support. I look forward to see the project grow and the upcoming update.
Just in case I copied over all the files in the Toshiba 1TB drive and then I simply tried to run Rescuezilla again. It worked perfectly this time! So it seems that as you suspected there was an underlying issue with the hard drive for some reason. It's strange that it worked normally up until now, but hopefully it will continue to work normally.
That's great news! Another user has recently reported Rescuezilla v1.0.6.1 encountering an error 2.5 hours into the backup process of their largest partition (also NTFS). I speculate that the backup of hundreds of gigabytes in a large and sequential transfer is exposing pre-existing bad sectors, which partclone is not handling well.
Some people reading this may think that Rescuezilla v1.0.6.1 could contain a race condition. To this I note that Rescuezilla v1.0.6.1 continues the Redo Backup and Recovery architecture of being a single threaded Perl application where activity that appears to be running in parallel is achieved by launching subprocesses (such as 'partclone') which only communicate by the graphical interface processing its standard text streams. While this is an awful architecture for a variety of reasons, the fact the application is single threaded means that a "race condition" internal to Rescuezilla v1.0.6.1 is ruled out.
As a little feedback on the use of Rescuezilla, it was not entirely clear when I could close the window. The little prompt at the end telling me how long it took made it fairly obvious, but still an option to "go back" or something more clear to know that the program is done. No biggy just my opinion (maybe a little too paranoid whether it worked or not at this point).
The fact that users are unable to easily cancel the Rescuezilla backup process is definitely a very big issue that I am aware of. This long-standing issue stems can be easily fixed when the soon to be completed command pipeline exit code handling (#29) task changes how the command pipeline subprocesses are handled. Rescuezilla v1.0.6.1 does not even try to exit partclone by sending it a terminate signal. I have updated the task to reflect mention this.
Also, I am in the process of adding confirmation and summary pages to the graphical interface.
First of all, the hard drive had about half of unallocated space but at the prompt I could select which partitions I wanted to backup, and it didn't appear to me that this space was included. Currently the backed up folder sits at 130GB. Could this be a problem if I attempt to recover the system into a 500GB drive (which is about the space that is currently taking)?
So Rescuezilla v1.0.6.1 does not support restoring to a drive that's smaller than the original even if there's plenty of unallocated space.
Early on during the development of Rescuezilla, I imported some work by another author which added a "restore to disk smaller than original" feature to Redo Backup and Recovery. I have since found some issues with the approach that author took when used with GPT partition tables (which your drive contains), so I have completely de-emphasized any "restore to disk smaller than original" functionality. I will eventually be taking another look at it, but at this stage it's likely that even Rescuezilla v1.0.7 will not add this feature.
And also, are there any risks to the PC if I attempt to recover from the files taken?
For reference I'm attaching the files for the latest, successful backup in case there's something there that helps. Please let me know if you'd like to see something about the others.
There is no risk in restoring the working backup that you have made. I have examined your most recent logs and can see no issues. Given your most recent Rescuezilla backup reported completely successful operation, I am very comfortable in its ability to successfully be restored.
I should re-iterate that Rescuezilla v1.0.6.1 does not provide an ability to restore partitions individually, so it will restore the entire backup. This is not a problem for you as your disk does not contain a dual-boot environment where you would want to retain some partitions and restore others. Also I should note that Rescuezilla v1.0.6.1 always overwrites the partition table on restore, so if you add any partitions to the end of the disk and the information that a partition existed this will be overwritten during a restore.
Thank you once again for all your quick and kind support. I look forward to see the project grow and the upcoming update.
No problem! Thank YOU for being very helpful with my debugging efforts, and for having used Rescuezilla v1.0.6.1!
Last edit: Rescuezilla 2020-07-23
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This is not a problem for you as your disk does not contain a dual-boot environment where you would want to retain some partitions and restore others.
Actually the reason I ran into Rescuezilla is because I want to setup a dual-boot system with Linux. Since it's for a friend, I'd like to have the option to revert back to it's current state should I ever need to. In fact, if it comes to that, overwriting any partitions and data at the end of the disk is what I prefer but I'm sure things will run smoothly :)
Cheers!
👍
1
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, I was trying to make a backup of my hard-drive but the process failed an hour later at around 90% of the way. The error showed "Input/Output error", and after that another annoying box kept popping up without giving me the option to start over, report bug or even exit the application. I had to kill the process from the terminal. I'll attach screenshots of these errors.
I restarted the PC and made sure everything still worked fine. It did, but on the external drive (where I would store the backup from my pc) there was the folder that I named "backup" which I can't access nor delete, it says is corrupted or damaged... any help on how to get rid of it is much appreciated.
I tried again later after that with similar results. The backup process failed at around 20%, but this time the backup folder on the external drive can be accessed and shows a bunch of files and logs which I'll attach here.
For reference, the drive I'm trying to backup is 1TB (size shows 931.51GB) and contains 4 partitions. However only about half the space is taken by these partitions, and the rest remains unallocated. In the log ouput there are three drives: sda is PC I want to backup, sdb is the live rescuezilla usb and sdc is where I want to store the backup.
Thank you in advance for your help, any ideas are very much appreciated!
Hi yanzen,
Thanks for attempting to use Rescuezilla!
So what has happened here is part way through the Rescuezilla v1.0.6.1 backup process, the command pipeline that is conducting the backup has failed for some reason. Usually this is because the "partclone" utility that is internally being used is unable to understand the filesystem that is undergoing the backup process. Though for some users the root cause can vary: eg, instability in accessing the destination network share folder, running out of RAM or a variety of other reasons. The log your provided indicates the "partclone failed partially through the backup" but little else. As you have seen the unexpected exit of a component within the Rescuezilla command pipeline causes current versions of Rescuezilla to break in ways that the user is not expecting (task #29), leading to the output of another part of the pipeline being gobbled up and immediately displaying it, leading to several incredibly poorly worded error messages. This issue causes a clearly terrible user experience.
One bright side is the fact something went wrong is always extremely clear when this very rare issue occurs: no user will ever think that their partially completed backup that has spewed several error messages was a success. As part of the Rescuezilla v1.0.7, I am finally completely fixing the the brittle command pipeline that has affected all Rescuezilla (and before that all Redo Backup and Recovery) as part of Rescuezilla v1.0.7 scheduled to be released in mid-October 2020. I have listed the poor exit code handling and error message under the "bug advisories" section on the Rescuezilla release page, but until now I have been focusing on other Rescuezilla tasks. I'm sorry I haven't corrected this poor usability issue yet.
Now, as far as the issue around a folder named 'backup', that is indeed very strange. It may be related to permissions. Could you try launching Rescuezilla, launching the File Manager from the desktop and clicking your drive on the left-hand menu to mount your drive. Are you at least able to navigate this folder using the Rescuezilla's file manager?
You may be able to delete this folder from the file manager, but you may need to right-click "Open current directory in terminal" to carefully delete it as a 'root' user.
If you are able to access the folder named 'backup', please upload the rescuezilla.log file from your first attempt, it may shed some light to why the 'partclone' utility is failing. By the way, I am also very interested in seeing the files ending with "partX_partclone.log" because they may contain some information not found in the main Rescuezilla log file.
I am very happy to continue to provide assistance for as long as required. Thanks again for attempting to use Rescuezilla! Sorry for any inconvenience caused by my lack of having completed task #29!
Hello and thank you for the quick reply!
I did as you suggested and opened the external drive from Rescuezilla and the files are all there, it seems is only the Windows machine that cannot read them. However, most of the files show in the description as "inode/x-corrupted type" (as opposed to "plain text document" for example) and cannot open any of them. From the terminal is clear that something is wrong with them. I'll attach a screenshot of this, and the "rescuezilla.log" that you requested.
On the second backup attempt all files are there and readable, I'll attach the log files requested. I've renamed them for convenience to tell which files belong to which backup attempt.
Thanks again for the help, and I'm really looking forward to that upcoming update!
Last edit: yanzen 2020-07-22
The issue with your Toshiba 1TB destination drive is very worrying. It looks like it's not simply an issue related to permissions but an indication that your backup drive has some filesystem corruption. I recommend that you do not use the Linux file manager to modify that filesystem until the filesystem is repaired.
It's possible that the force kill of the Rescuezilla process you mentioned has left the filesystem in an inconsistent state. The best way to fix this is to boot Windows and run
chkdskto do a filesystem check of the NTFS partition of your Toshiba 1TB drive. Here's achkdsktutorial I found. You may wish to initially run the filesystem check without the "fix errors" flag, so it scans in read-only mode as a sanity check without modifying the filesystem in any way.I examined the log files from your most recent reply and they contain no indications why the Rescuezilla command pipeline failed part way through the 'partclone.ntfs' command. Unfortunately Rescuezilla v1.0.6.1 does not log the exit codes from the Rescuezilla pipeline commands, nor the kernel diagnostic messages, which would both be very useful to debug this kind of issue.
I should mention also mention there is very real possibility that your Toshiba 1TB destination drive is developing issues for reasons unrelated to Rescuezilla. If your drive encounters eg, SATA write errors while making a backup, this could explain why the backup process froze and also why there is some corrupted files on the filesystem.
One way to test that hypothesis is to get another external harddrive unrelated to your Toshiba 1TB and try using Rescuezilla v1.0.6.1 to make a backup of your internal hard drive again. If this succeeds without issue it's an indication that it's your Toshiba 1TB has errors, rather than anything to do with Rescuezilla.
Another potentially helpful indicator is examine the kernel diagnostic messages for hard drive errors immediately after experiencing an error. To do this, open a terminal as the root user (again, by right-click "Open current directory in terminal") and type
dmesg -Tw. If a harddrive is failing then certain kinds of access will add entries to the dmesg output (highlighted in bright red) saying something like eg, ATA read has failed to because a bad sector.This issue may indicate a potential problem in Rescuezilla v1.0.6.1. I am very keen to get to the bottom of this issue with extremely high priority. Please be extra careful with your Toshiba 1TB drive until we have debugged exactly what is going on.
Last edit: Rescuezilla 2020-07-22
I didn't even consider the hard drive as a point of failure, but I'm glad you brought that up. The hard drive is actually fairly new, bought just a few months ago, although it may be that I haven't used it long enough to noticed an issues until now. I did as suggested and ran
chkdskwhich discovered an issue with the drive. Unfortunately the command prompt wasn't very helpful as to what the issue was, so I simply added the/fflag to automatically fix it. A final check and no errors found, and all files were recovered and readable again.Just in case I copied over all the files in the Toshiba 1TB drive and then I simply tried to run Rescuezilla again. It worked perfectly this time! So it seems that as you suspected there was an underlying issue with the hard drive for some reason. It's strange that it worked normally up until now, but hopefully it will continue to work normally.
As a little feedback on the use of Rescuezilla, it was not entirely clear when I could close the window. The little prompt at the end telling me how long it took made it fairly obvious, but still an option to "go back" or something more clear to know that the program is done. No biggy just my opinion (maybe a little too paranoid whether it worked or not at this point).
On recovering from the backup folder, I have a couple of questions. First of all, the hard drive had about half of unallocated space but at the prompt I could select which partitions I wanted to backup, and it didn't appear to me that this space was included. Currently the backed up folder sits at 130GB. Could this be a problem if I attempt to recover the system into a 500GB drive (which is about the space that is currently taking)? And also, are there any risks to the PC if I attempt to recover from the files taken?
For reference I'm attaching the files for the latest, successful backup in case there's something there that helps. Please let me know if you'd like to see something about the others.
Thank you once again for all your quick and kind support. I look forward to see the project grow and the upcoming update.
Cheers!
That's great news! Another user has recently reported Rescuezilla v1.0.6.1 encountering an error 2.5 hours into the backup process of their largest partition (also NTFS). I speculate that the backup of hundreds of gigabytes in a large and sequential transfer is exposing pre-existing bad sectors, which partclone is not handling well.
Some people reading this may think that Rescuezilla v1.0.6.1 could contain a race condition. To this I note that Rescuezilla v1.0.6.1 continues the Redo Backup and Recovery architecture of being a single threaded Perl application where activity that appears to be running in parallel is achieved by launching subprocesses (such as 'partclone') which only communicate by the graphical interface processing its standard text streams. While this is an awful architecture for a variety of reasons, the fact the application is single threaded means that a "race condition" internal to Rescuezilla v1.0.6.1 is ruled out.
The fact that users are unable to easily cancel the Rescuezilla backup process is definitely a very big issue that I am aware of. This long-standing issue stems can be easily fixed when the soon to be completed command pipeline exit code handling (#29) task changes how the command pipeline subprocesses are handled. Rescuezilla v1.0.6.1 does not even try to exit partclone by sending it a terminate signal. I have updated the task to reflect mention this.
Also, I am in the process of adding confirmation and summary pages to the graphical interface.
So Rescuezilla v1.0.6.1 does not support restoring to a drive that's smaller than the original even if there's plenty of unallocated space.
Early on during the development of Rescuezilla, I imported some work by another author which added a "restore to disk smaller than original" feature to Redo Backup and Recovery. I have since found some issues with the approach that author took when used with GPT partition tables (which your drive contains), so I have completely de-emphasized any "restore to disk smaller than original" functionality. I will eventually be taking another look at it, but at this stage it's likely that even Rescuezilla v1.0.7 will not add this feature.
There is no risk in restoring the working backup that you have made. I have examined your most recent logs and can see no issues. Given your most recent Rescuezilla backup reported completely successful operation, I am very comfortable in its ability to successfully be restored.
I should re-iterate that Rescuezilla v1.0.6.1 does not provide an ability to restore partitions individually, so it will restore the entire backup. This is not a problem for you as your disk does not contain a dual-boot environment where you would want to retain some partitions and restore others. Also I should note that Rescuezilla v1.0.6.1 always overwrites the partition table on restore, so if you add any partitions to the end of the disk and the information that a partition existed this will be overwritten during a restore.
No problem! Thank YOU for being very helpful with my debugging efforts, and for having used Rescuezilla v1.0.6.1!
Last edit: Rescuezilla 2020-07-23
Actually the reason I ran into Rescuezilla is because I want to setup a dual-boot system with Linux. Since it's for a friend, I'd like to have the option to revert back to it's current state should I ever need to. In fact, if it comes to that, overwriting any partitions and data at the end of the disk is what I prefer but I'm sure things will run smoothly :)
Cheers!