Request for help to test SnapRAID v12.0 RC5

[Andrea Mazzoleni]

Hi,

I've prepared a Release Candidate of SnapRAID v12.0 RC5. It's available at: http://beta.snapraid.it/

It supports parallel disk scanning to speedup the initial phase of the 'diff' and 'sync' commands.

If you have a spare array, you can help to test this new functionality and see how much does it improve the initial scan phase. (the one called "Scanning..." in sync). Now the disks are all scanned in parallel, and the process should be as long as the longest disk to read.

The 'diff' command is safe, as it's read-only. The 'sync' command is expected to be safe, but it's a good precaution to not use it on a production system until the release.

Thanks,
Andrea

[Leifi Plomeros]

The output from snaprad diff in snapraid-12.0-rc-5-g2c3f9b8-windows-x64.zip is a bit messed up.

C:\Snapraid>snapraid.exe diff --test-fmt disk
Loading state from C:/snapraid/snapraid.content...
WARNING! With 28 disks it's recommended to use four parity levels.
Comparing...
addadd add "EC IA1:TV/9-1EBadd EB5_I:TV/9-1-1/9-1-1.s05eadd EB3_H:TV/9-1-1/

...

The total number of lines (243) is correct and the content of some of the lines looks OK but most are corrupted:

1.H.264d DD-a" <-- That is a very small part of a file name on a single row
PFLEa5dUBdd X.3_H.1dE:m. B7kHEBv_TV2 <-- The following row not even recognizable as part of file name.

I haven't tested anything else yet (but I will probably do it later this week).

[Andrea Mazzoleni]

Thanks Leifi!

I prepared a new RC6 that fixes this Windows issue: http://beta.snapraid.it/

[Charles Grisamore]

I did a quick test and didn't really notice a faster scan however I only have four data drives (8TB each) and I expect the benefit of parallel scanning is much greater with more drives in the array.

[Leifi Plomeros]

I finaly got around to testing it.

With all drives already spinning I couldn't observe any significant speed increase from snapraid diff.

When running sync all disks were scanned in 0 seconds so maybe I would need more files per disk to really notice the difference or do a test where all drives are sleeping.

The output issue is definitely fixed in any case.

[Ivan NM.]

8x14/16TB data drives

diff summary:

  433555 equal
     715 added
      21 removed
      16 updated
     351 moved
       1 copied
       0 restored
There are differences!

Times checked with Timer utility from 7-zip author.

Snapraid 11.6:

Kernel  Time =    14.015 =    5%
User    Time =    37.593 =   15%
Process Time =    51.609 =   21%    Virtual  Memory =   6950 MB
Global  Time =   236.085 =  100%    Physical Memory =   6922 MB

Snapraid 12 RC:

Kernel  Time =    13.609 =   12%
User    Time =    46.281 =   41%
Process Time =    59.890 =   53%    Virtual  Memory =   6951 MB
Global  Time =   112.750 =  100%    Physical Memory =   6925 MB

[Master CATZ]

massive speed increase from 11520 seconds to 116 seconds during initial startup over 96 drives
so that is 3 hrs saved !!!

now I just have to work out what is bottle necking my syncing that seems to be hitting a 20MB/s write speed wall while writing to the parity drives ~ 1/4 their normal sync write speeds
it could be a possibility that the files being synced when i was watching it were only coming from a few drives as well as it only takes a few large files to throw out the entire distribution

started 3366117 MB sync 18∶58∶29 and it finished 01∶02∶52
but 6 hrs to do 3 tb is quite reasonable

I might actually do syncs more often now that it only takes 2 mins to get past the startup stages ,
as I can actually do a quick diff :P