Menu ▾ ▴

snapraid sync "file errors"

Help
Kevin Mychal M. Ong
2022-02-12
2022-02-20

  • Kevin Mychal M. Ong

    Kevin Mychal M. Ong - 2022-02-12

    So I added a couple of drives to my mergerfs filesystem (which snapraid is monitoring). Everything was good the other day when I ran snapraid sync but when I ran it again now, here's what I got:

    root@epsilon:~# snapraid sync
Self test...
Loading state from /var/snapraid.content...
WARNING! With 5 disks it's recommended to use two parity levels.
Scanning...
Scanned d2 in 0 seconds
Scanned d3 in 0 seconds
Scanned d5 in 0 seconds
Scanned d4 in 0 seconds
Scanned d1 in 0 seconds
Using 843 MiB of memory for the file-system.
Initializing...
Resizing...
Saving state to /var/snapraid.content...
Saving state to /mnt/disk1/.snapraid.content...
Saving state to /mnt/disk2/.snapraid.content...
Saving state to /mnt/disk3/.snapraid.content...
Saving state to /mnt/disk4/.snapraid.content...
Saving state to /mnt/disk5/.snapraid.content...
Saving state to /mnt/parity1/.snapraid.content...
Verifying...
Verified /mnt/disk3/.snapraid.content in 0 seconds
Verified /mnt/disk4/.snapraid.content in 0 seconds
Verified /mnt/disk1/.snapraid.content in 0 seconds
Verified /var/snapraid.content in 0 seconds
Verified /mnt/disk2/.snapraid.content in 0 seconds
Verified /mnt/parity1/.snapraid.content in 0 seconds
Verified /mnt/disk5/.snapraid.content in 0 seconds
Using 96 MiB of memory for 64 cached blocks.
Selecting...
Syncing...
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/PhotoDirector/898c4d39831f90a1288b65041277a311/System.ni.dll_0.aux' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/PhotoDirector/898c4d39831f90a1288b65041277a311/System.ni.dll_2.aux' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Gradient/Microsoft.VisualStudio.Tools.Office.AppInfoDocument.v9.0.ni.dll' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Gradient/Microsoft.VisualStudio.Tools.Office.AppInfoDocument.v9.0.ni.dll' at position '1'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Gradient/Microsoft.VisualStudio.Tools.Office.AppInfoDocument.v9.0.ni_1.dll' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Gradient/Microsoft.VisualStudio.Tools.Office.AppInfoDocument.v9.0.ni_1.dll' at position '1'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Gradient/Microsoft.VisualStudio.Tools.Office.AppInfoDocument.v9.0.ni_0.dll' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Gradient/Microsoft.VisualStudio.Tools.Office.AppInfoDocument.v9.0.ni_0.dll' at position '1'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Gradient/Microsoft.VisualStudio.Tools.Office.AppInfoDocument.v9.0.ni_2.dll' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Gradient/Microsoft.VisualStudio.Tools.Office.AppInfoDocument.v9.0.ni_2.dll' at position '1'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/3220014388.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/skin_en-GB_female_TTS_1.lua' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/qsvg.dll' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/2672157974_2.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/462006811.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/2672157974_0.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/462006811_0.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#3 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/2672157974_1.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk5/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/PhotoDirector/898c4d39831f90a1288b65041277a311/System.ni.dll.aux' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk3/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/skin_en-GB_female_TTS.lua' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk3/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/3547987947_0.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk3/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/3547987947_1.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk3/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/skin_en-GB_female_TTS_0.lua' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk3/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/16832737.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk3/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/2672157974.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk3/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/2672157974_3.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk3/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/3547987947.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
Data change at file '/mnt/disk3/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/Camera/en-GB_female_TTS/2672157974_4.pri' at position '0'
WARNING! Unexpected data modification of a file without parity!
Try removing the file from the array and rerun the 'sync' command!
100% completed, 25 MB accessed in 0:00

     d1  9% | *****
     d2  0% |
     d3  0% |
     d4 86% | ****************************************************
     d5  0% |
 parity  0% |
   raid  0% |
   hash  2% | *
  sched 10% | ******
   misc  0% |
            |______________________________________________________________
                           wait time (total, less is better)


      28 file errors
       0 io errors
       0 data errors
WARNING! Unexpected file errors!
Saving state to /var/snapraid.content...
Saving state to /mnt/disk1/.snapraid.content...
Saving state to /mnt/disk2/.snapraid.content...
Saving state to /mnt/disk3/.snapraid.content...
Saving state to /mnt/disk4/.snapraid.content...
Saving state to /mnt/disk5/.snapraid.content...
Saving state to /mnt/parity1/.snapraid.content...
Verifying...
Verified /mnt/disk3/.snapraid.content in 0 seconds
Verified /mnt/disk2/.snapraid.content in 0 seconds
Verified /mnt/parity1/.snapraid.content in 0 seconds
Verified /mnt/disk4/.snapraid.content in 0 seconds
Verified /mnt/disk1/.snapraid.content in 0 seconds
Verified /var/snapraid.content in 0 seconds
Verified /mnt/disk5/.snapraid.content in 0 seconds

    What does "WARNING! Unexpected data modification of a file without parity!" mean? Shouldn't a sync command sync everything that is not synced and add to the parity?

     

  • Kevin Mychal M. Ong

    Kevin Mychal M. Ong - 2022-02-12

    I think this happened after I ran snapraid touch and yes I did rebalance the whole mergerfs array so that all drives (old and new) have approximately equal usage in terms of percentage.

     

    Last edit: Kevin Mychal M. Ong 2022-02-12

  • Kevin Mychal M. Ong

    Kevin Mychal M. Ong - 2022-02-13
     

  • Kevin Mychal M. Ong

    Kevin Mychal M. Ong - 2022-02-14

    Can anybody help me out here? Thanks.

     

  • Andrea Mazzoleni

    Andrea Mazzoleni - 2022-02-15

    Hi Kevin,

    Try running sync with the --force-nocopy option.

    Ciao,
    Andrea

     

  • Kevin Mychal M. Ong

    Kevin Mychal M. Ong - 2022-02-16

    Hi Andrea,

    That fixed the issue. Thanks! However, out of curiosity do you know what the root cause of the issue was?

    Regards,
    Kevin

     

  • Andrea Mazzoleni

    Andrea Mazzoleni - 2022-02-17

    Hi Kevin,

    SnapRAID was thinking that such files were copies of other files, but the hash was not matching the copies. But it's not clear why this happened.

    Ciao,
    Andrea

     

  • Kevin Mychal M. Ong

    Kevin Mychal M. Ong - 2022-02-17

    Hi Andrea,

    I see. Right, I didn't really do any copying of files. The mergerfs rebalance script just moves files to the drives in the pool using rsync. Could this be because of the file types or something?

    Also, since one of my drives since to start developing bad sectors, I need to change it from a 3TB drive to a 6TB drive. What is the recommended way of doing this?

    Thanks.

    Kevin

     

  • Andrea Mazzoleni

    Andrea Mazzoleni - 2022-02-17

    Hi Kevin,

    Did that rsync run while sync was in progress the first time you got that error ?

    If yes, it could be the cause.

    To replace a disk see: http://www.snapraid.it/faq#repdatadisk

    Ciao,
    Andrea

     

  • Kevin Mychal M. Ong

    Kevin Mychal M. Ong - 2022-02-17

    Hi Andrea,

    No, I made sure snapraid Is not running when rsync was in progress. If I remember correctly, it wasn't until I issued a touch command did this issue with a couple of files appear.

    Thanks for the link in replacing a disk.

    Regards,
    Kevin

     

  • Kevin Mychal M. Ong

    Kevin Mychal M. Ong - 2022-02-19

    Hi Andrea,

    After copying the files from the old drive to the new drive and running snapraid diff, I get this:

    root@epsilon:~# snapraid diff
Loading state from /var/snapraid.content...
UUID change for disk 'd2' from 'e53710c7-4994-42de-aa78-9e05df30ae67' to '85e786df-fd8d-4b18-9f25-2cb9069c81af'
WARNING! With 5 disks it's recommended to use two parity levels.
Comparing...
WARNING! UUID is changed for disks: 'd2'. Not using inodes to detect move operations.

  179133 equal
       0 added
       0 removed
       0 updated
       0 moved
       0 copied
       0 restored
No differences

    The UUID change is, of course, expected but what does the "not using inodes to detect move operations" warning message signify? Is it something that we need to worry about?

    Regards,
    Kevin

     

  • Kevin Mychal M. Ong

    Kevin Mychal M. Ong - 2022-02-20

    So after running snapraid, those errors went away. I rebalanced my drives again using the mergerfs rebalance script. After that, I ran sync and got this:

    Data change at file '/mnt/disk2/backup/Laptop Backup - Irene/#2 Lost Partition(NTFS)/Huawei Phone_IRENE/PhotoDirector/898c4d39831f90a1288b65041277a311/System.ni.dll.aux' at position '0'
WARNING! Unexpected data modification of a file without parity!
This file was detected as a copy of another file with the same name, size,
and timestamp, but the file data isn't matching the assumed copy.
If this is a false positive, and the files are expected to be different,
you can 'sync' anyway using 'snapraid --force-nocopy sync'

    So that's the exact same error as before. It happens after I do a rebalance. If it helps, this is the command used by the balance script:

    rsync -avlHAXWE --relative --progress --remove-source-files <destination dir=""></destination>

    And of course, after running snapraid --force-nocopy sync, the problem is fixed.

     

    Last edit: Kevin Mychal M. Ong 2022-02-20

Log in to post a comment.