Hello
I have begun using Snapraid after Flexraid no longer works on my system,and so far it seems very fast,and stable,however i have run in to a little "problem" i was hoping you could help me with.
I use snapraid on my 50TB home server where i store all my movies,videos,music and photos,and thus snapraid is seemingly perfect,however, during a sync it has restored and overwritten a file it shouldnt have, how do i stop this from happening?
I had replaced an srt subtitle file of same name and same size with a new better one, and the new one had been deleted to be replaced by the restored old one,this i do not want happening on anything at all..
Thank you
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Are you sure that was "sync" you ran and not "fix"? sync should have updated the array to include the new version, where fix would have rolled back to the previous version. Both presuming the modified file had not yet been synced.
Updating or deleting files in an existing array can be dangerous. Your array is no longer synced and you may not be able to recover from damage.
Either sync immediately after any update or delete, or what I do: on each disc, keep a "pending" and a "delete" folder outside of the array directores. Updates go in the pending folder. Deleted files are moved to the delete folder.
Just before my weekly sync, I move any files in the pending folders to their place in the array. After the sync, any files in the delete folders can be actually deleted.
This minimizes the time window of risk.
New files added to the array directories are not a problem. They are invisible to snapraid until they are synced, although you can see them with "diff".
-Bill
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am certain i only ran a sync.. I havent tried a fix yet at all,so have no idea how that works..The log also says restore and then the file name,and this is what bothers me..
I run sync EVERY time i add,delete,change or move stuff around my array..
I dont do pooling,so all drives have their own drive letter.
So how can snapraid make the decision to overwrite a new file? i run the snapraid sync -v -l c:/sync.log like this
this is the entry in the log scan:restore:d12:Conan The Barbarian (2011)/Conan The Barbarian (2011).srt
summary:equal:243541
summary:moved:8
summary:copied:13
summary:restored:1
summary:updated:4
summary:removed:1174
summary:added:5847
summary:exit:diff
memory:used:7373978385
summary:error_readwrite:0
summary:error_data:0
summary:exit:ok
Last edit: Benny King 2014-06-01
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
SnapRAID never write data in sync, only parity. 100% sure.
The "restore" entry, means that SnapRAID detected that someone other restored this file, meaning that this file had an inode change, without a corresponding timestamp change.
This something that backup programs do, but also other programs may have this behavior.
Ciao,
Andrea
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am not sure what you mean by someone else? I have nothing running on this box at all,i have disabled ALL non windows functions i do not use,and the only other program running is hd sentinel which monitors drive health..
there is no backup programs,no anti virus,no firewalls,no nothing.
So what could write this file? Like i said,i deleted the old srt file,and replaced it with this new one prior to sync,so how can my new srt file be overwritten?
For the record,i use YAMJ and YADIS,both are on other machines on the network,and do not access these files unless i run the specific files from my media streamers
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
He means that some program other than SnapRAID must have restored the file. It is no use asking anyone on the forum what could have written to your drive. You have to figure it out for yourself. But it could NOT have been snapraid sync that did it, since sync never modifies the data files.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
But why does snapraid mark it as "restored" then,if snapraid didnt restore it? It makes no sense.. Also, there is nothing on my box that could restore the file.
Also, the file was the new one BEFORE the sync, and it was replaced AFTER The sync, snapraid was the only program running.
So you see where i might get confused..
What determines the "restored" status? And wouldnt it be better to call this something else besides restored? Maybe call it "changed" if that fits what snapraid does better?
What can i do to prevent "restored" from happening again?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Snapraid did not "mark" the file restored, or do anything to modify the file. It just scanned the file and noted that it appears to be "restored" (Andrea already explained the criteria for that).
Anyway, you need to listen to what Andrea and I are telling you. snapraid sync did not modify your file. You need to figure out for yourself what did it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
And if i knew what an "inode change" was i wouldnt be asking how to prevent this.
So i will just delete snapraid again,and not use it,clearly it is too advanced for me..
Goodbye
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK,finally we are getting somewhere,so why didnt it tag the other files i deleted the same way? Which factors detemine if a file is restored,new or moved/copied?? I replaced the old mkv with a filesize of about 12GB with a different one named exactly the same but with a filesize of 30GB,was this too large a discrepency for snapraid to mark it restored too and thus marked it new?
Andrea,you still might want to change the naming from "restore" to "changed" or similar,i can't be the only one confused by this.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
All the filesystems associate to files an internal identifier that doesn't change for all the file existence. It's usually called inode.
If the inode doesn't change, SnapRAID assume that the file is "moved", because in move operations the inode never changes. If the inode is different, the file is assumed "restored" if it's in the same location as before, or "copied" if in a different location.
Yep. "restored" may be misleading. I have to change it to something else, but not sure what's a good replacement.
Ciao,
Andrea
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It seems there are not very many operations that result in a file with the same path and filename, same timestamp, but a different inode. You named one -- restoring a file from a backup. I think some synchronization programs may do it. I'm not aware of another example. I know that just overwriting a file will NOT do it.
As a test, I just tried an overwrite operation in linux, using either cp or mv. With cp, the inode remained the same and the timestamp changed. With mv, both the inode and the modify timestamp changed to that of the source file to the overwrite (and the change time updated to the time of the overwrite).
In Windows, doing a "move and replace" overwrite, the timestamp changed to that of the source file to the overwrite. Doing a "copy and replace" overwrite, the modify time changed (although the creation time remained the same).
Since different inode / same timestamp would ordinarily only occur when restoring or synchronizing a file from backup, it seems that "restored" is a good name. The only thing more accurate I can think of is "inode changed". But I think that many users would not know what that meant.
It is not clear what Benny did to get a file with different inode but the same timestamp. I guess if he deleted the old file, and then moved a backup copy of the file into the same filename as the old file, that could result in different inode / same modify timestamp. But in that case, restore is a reasonable name for what happened, since if the files were different, you would expect the timestamp to be different.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
jwill
You seem to be mixing something up. If you read my posts,and this is why i was confused.. I had a folder with a movie file .mkv,this folder also holds a subtitle file .srt and an nfo,and 2 jpgs. now, what i did was DELETE the mkv,and the srt file from the snapraid server which i done a million times before both in flexraid and snapraid, then i moved 2 new files from a networked pc into this folder,a new srt and a new mkv. The files were named exactly the same, and the new mkv was about 10GB larger,and the new srt file was same size,now the dates on these files were newer than the ones i deleted.
To me this does NOT constitute a "restore" but a "changed" status,as i did NOT copy or move a backup file of the old one,but a total new replacement file with different date,and possibly size (not sure here,as subtitle files tend to be small kb files)
Now, you also state snapraid does no writing when doing a sync, but somehow my new srt was replaced by the old srt, and how this happened is a mystery,as there is no program whatsoever on the box that could do this,so i have absolutly no idea how that happened.
I still think restore is a very confusing name,because it suggests that snapraid has replaced my file with one in parity
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello
I have begun using Snapraid after Flexraid no longer works on my system,and so far it seems very fast,and stable,however i have run in to a little "problem" i was hoping you could help me with.
I use snapraid on my 50TB home server where i store all my movies,videos,music and photos,and thus snapraid is seemingly perfect,however, during a sync it has restored and overwritten a file it shouldnt have, how do i stop this from happening?
I had replaced an srt subtitle file of same name and same size with a new better one, and the new one had been deleted to be replaced by the restored old one,this i do not want happening on anything at all..
Thank you
Are you sure that was "sync" you ran and not "fix"? sync should have updated the array to include the new version, where fix would have rolled back to the previous version. Both presuming the modified file had not yet been synced.
Updating or deleting files in an existing array can be dangerous. Your array is no longer synced and you may not be able to recover from damage.
Either sync immediately after any update or delete, or what I do: on each disc, keep a "pending" and a "delete" folder outside of the array directores. Updates go in the pending folder. Deleted files are moved to the delete folder.
Just before my weekly sync, I move any files in the pending folders to their place in the array. After the sync, any files in the delete folders can be actually deleted.
This minimizes the time window of risk.
New files added to the array directories are not a problem. They are invisible to snapraid until they are synced, although you can see them with "diff".
-Bill
I am certain i only ran a sync.. I havent tried a fix yet at all,so have no idea how that works..The log also says restore and then the file name,and this is what bothers me..
I run sync EVERY time i add,delete,change or move stuff around my array..
I dont do pooling,so all drives have their own drive letter.
So how can snapraid make the decision to overwrite a new file? i run the snapraid sync -v -l c:/sync.log like this
this is the entry in the log scan:restore:d12:Conan The Barbarian (2011)/Conan The Barbarian (2011).srt
summary:equal:243541
summary:moved:8
summary:copied:13
summary:restored:1
summary:updated:4
summary:removed:1174
summary:added:5847
summary:exit:diff
memory:used:7373978385
summary:error_readwrite:0
summary:error_data:0
summary:exit:ok
Last edit: Benny King 2014-06-01
Hi Benny,
SnapRAID never write data in sync, only parity. 100% sure.
The "restore" entry, means that SnapRAID detected that someone other restored this file, meaning that this file had an inode change, without a corresponding timestamp change.
This something that backup programs do, but also other programs may have this behavior.
Ciao,
Andrea
I am not sure what you mean by someone else? I have nothing running on this box at all,i have disabled ALL non windows functions i do not use,and the only other program running is hd sentinel which monitors drive health..
there is no backup programs,no anti virus,no firewalls,no nothing.
So what could write this file? Like i said,i deleted the old srt file,and replaced it with this new one prior to sync,so how can my new srt file be overwritten?
For the record,i use YAMJ and YADIS,both are on other machines on the network,and do not access these files unless i run the specific files from my media streamers
He means that some program other than SnapRAID must have restored the file. It is no use asking anyone on the forum what could have written to your drive. You have to figure it out for yourself. But it could NOT have been
snapraid syncthat did it, since sync never modifies the data files.But why does snapraid mark it as "restored" then,if snapraid didnt restore it? It makes no sense.. Also, there is nothing on my box that could restore the file.
Also, the file was the new one BEFORE the sync, and it was replaced AFTER The sync, snapraid was the only program running.
So you see where i might get confused..
What determines the "restored" status? And wouldnt it be better to call this something else besides restored? Maybe call it "changed" if that fits what snapraid does better?
What can i do to prevent "restored" from happening again?
Snapraid did not "mark" the file restored, or do anything to modify the file. It just scanned the file and noted that it appears to be "restored" (Andrea already explained the criteria for that).
Anyway, you need to listen to what Andrea and I are telling you.
snapraid syncdid not modify your file. You need to figure out for yourself what did it.And if i knew what an "inode change" was i wouldnt be asking how to prevent this.
So i will just delete snapraid again,and not use it,clearly it is too advanced for me..
Goodbye
You are still not listening. If you (or a script) did not run
snapraid fix, then it is 100% certain that snapraid did not modify your data file.Therefore, something other than snapraid modified your data file. Deleting snapraid will not solve that problem.
Hi Benny,
OK. This explains the "restored" entry. SnapRAID detected that you manually deleted and then rewritten this file.
You operated just like a backup program that restored this file from a previous copy, and SnapRAID is just reporting this info.
Ciao,
Andrea
Last edit: Andrea Mazzoleni 2014-06-02
OK,finally we are getting somewhere,so why didnt it tag the other files i deleted the same way? Which factors detemine if a file is restored,new or moved/copied?? I replaced the old mkv with a filesize of about 12GB with a different one named exactly the same but with a filesize of 30GB,was this too large a discrepency for snapraid to mark it restored too and thus marked it new?
Andrea,you still might want to change the naming from "restore" to "changed" or similar,i can't be the only one confused by this.
Hi Benny,
All the filesystems associate to files an internal identifier that doesn't change for all the file existence. It's usually called inode.
If the inode doesn't change, SnapRAID assume that the file is "moved", because in move operations the inode never changes. If the inode is different, the file is assumed "restored" if it's in the same location as before, or "copied" if in a different location.
Yep. "restored" may be misleading. I have to change it to something else, but not sure what's a good replacement.
Ciao,
Andrea
It seems there are not very many operations that result in a file with the same path and filename, same timestamp, but a different inode. You named one -- restoring a file from a backup. I think some synchronization programs may do it. I'm not aware of another example. I know that just overwriting a file will NOT do it.
As a test, I just tried an overwrite operation in linux, using either cp or mv. With cp, the inode remained the same and the timestamp changed. With mv, both the inode and the modify timestamp changed to that of the source file to the overwrite (and the change time updated to the time of the overwrite).
In Windows, doing a "move and replace" overwrite, the timestamp changed to that of the source file to the overwrite. Doing a "copy and replace" overwrite, the modify time changed (although the creation time remained the same).
Since different inode / same timestamp would ordinarily only occur when restoring or synchronizing a file from backup, it seems that "restored" is a good name. The only thing more accurate I can think of is "inode changed". But I think that many users would not know what that meant.
It is not clear what Benny did to get a file with different inode but the same timestamp. I guess if he deleted the old file, and then moved a backup copy of the file into the same filename as the old file, that could result in different inode / same modify timestamp. But in that case, restore is a reasonable name for what happened, since if the files were different, you would expect the timestamp to be different.
jwill
You seem to be mixing something up. If you read my posts,and this is why i was confused.. I had a folder with a movie file .mkv,this folder also holds a subtitle file .srt and an nfo,and 2 jpgs. now, what i did was DELETE the mkv,and the srt file from the snapraid server which i done a million times before both in flexraid and snapraid, then i moved 2 new files from a networked pc into this folder,a new srt and a new mkv. The files were named exactly the same, and the new mkv was about 10GB larger,and the new srt file was same size,now the dates on these files were newer than the ones i deleted.
To me this does NOT constitute a "restore" but a "changed" status,as i did NOT copy or move a backup file of the old one,but a total new replacement file with different date,and possibly size (not sure here,as subtitle files tend to be small kb files)
Now, you also state snapraid does no writing when doing a sync, but somehow my new srt was replaced by the old srt, and how this happened is a mystery,as there is no program whatsoever on the box that could do this,so i have absolutly no idea how that happened.
I still think restore is a very confusing name,because it suggests that snapraid has replaced my file with one in parity
I am not mixed up at all. Your description is the problem.
You claimed that snapraid modified your file during a sync. That is incorrect.
You claimed that your file was reverted to an old copy, but neither you nor some other program did it. That is incorrect.
Benny:
What programs do you run to manage your media library? Plex, MCM, MediaBrowser, XBMC, Yamm, Meta Browser, etc.?