Activity for Allyn Malventano
-
Allyn Malventano modified a comment on discussion Help
If a drive has just one folder present in its SnapRAID path and the contents within that folder were modified, SnapRAID throws a warning and fails to begin sync, as it believe that 'all the files present have been rewritten' when in reality that is not the case. Yes, the only directory present in the root had its attributes changed, but the contents of the drive are not all rewritten (in my example only a few files were moved around and none were deleted at all). Since SnapRAID had already scanned...
-
If a drive has just one folder present in its SnapRAID path and the contents within that folder were modified, SnapRAID throws a warning and fails to begin sync, as it believe that 'all the files present have been rewritten' when in reality that is not the case. Yes, the only directory present in the root had its attributes changed, but the contents of the drive are not all rewritten (in my example only a few files were moved around and none were deleted at all). Since SnapRAID had already scanned...
-
I've tested this on Windows but not yet on Linux. I suspect the behavior is similar: Delete a file. Recover deleted file with 'snapraid fix -f FILE'. Note file date and time reflect the original date/time at the time of the last sync. The above is expected, however this only works properly for deleted files: Modify a file. Overwrite the modified file, restoring its prior state with 'snapraid fix -f FILE'. Note the newer (revised) file date and time remains, even though the contents have reverted...
-
I've tested this on Windows but not yet on Linux. I suspect the behavior is similar: 1. Delete a file. 2. Recover deleted file with 'snapraid fix -f FILE'. 3. Note file date and time reflect the original date/time at the time of the last sync. The above is expected, however this only works properly for deleted files: 1. Modify a file. 2. Overwrite the modified file, restoring its prior state with 'snapraid fix -f FILE'. 3. Note the newer (revised) file date and time remains, even though the contents...
-
I've tested this on Windows but not yet on Linux. I suspect the behavior is similar: Delete a file. Recover deleted file with 'snapraid fix -f FILE'. Note file date and time reflect the original date/time at the time of the last sync. The above is expected, however this only works properly for deleted files: Modify a file. Overwrite the modified file, restoring its prior state with 'snapraid fix -f FILE'. Note the newer (revised) file date and time remains, even though the contents have reverted...
-
Given the long rebuild times for a missing/failed disk, some files on the remaining disks might be moved or relocated during the rebuild. When SnapRAID runs into a file that was relocated after the rebuild started, it aborts with the following message: DANGER! file '(file name here)' disappeared. If you moved it, please rerun the same command. I understand the caution and the desire to abort for this case, but if there was sufficient additional parity to continue even with a missing file, is there...
-
There are a few other read/write operations that I've noted to not 'line up' with 1MB block devices: Loading state from snapraid.content (even from SSD this appears slower than it should be - likely 128K reads based on my observations). When using blocksize 1024, data disk reads appear to be aligned, however parity disks are seeing ~20% additional write requests than expected to the disk, suggesting that parity file writes might not be aligned to the configured blocksize. Is there some header information...
-
There are a few other read/write operations that I've noted to not 'line up' with 1MB block devices: Loading state from snapraid.content (even from SSD this appears slower than it should be - likely 128K reads). When using blocksize 1024, data disk reads appear to be aligned, however parity disks are seeing ~20% additional write requests than expected to the disk, suggesting that parity file writes might not be aligned to the configured blocksize. Is there some header information in the parity files...
-
I have a 3.7GB snapraid.content file saving to a zfs volume with a 1MB blocksize and block caching disabled (caching metadata only). I've noted that saving state to this file is fast (drive speed), but verifying takes a very ling time: Verified /z/p04/snapraid.content in 314 seconds The drive reads at ~200MB/s. This should take ~ 20 seconds. Watching the drive status (zpool iostat) while verifying, I can observe a steady read rate from the drive for the whole 5+ minutes. It appears that the snapraid.content...
-
Data rate for an array running on this system was only 1356 MB/s, with the following output: ^C%, 31045822 MB, 1356 MB/s, CPU 85%, 21:02 ETA Stopping for interruption at block 19055567 23% completed, 31046803 MB accessed in 6:31 h01 1% | h02 1% | h03 1% | h04 1% | h05 0% | h06 0% | h07 0% | (...) d33 0% | d34 0% | d35 0% | parity 0% | 2-parity 0% | 3-parity 0% | 4-parity 0% | raid 59% | *********************************** hash 19% | *********** sched 2% | * misc 0% | wait time (total, less is better)...
-
Data rate for an array running on this system was only 1356 MB/s, with the following output: ^C%, 31045822 MB, 1356 MB/s, CPU 85%, 21:02 ETA Stopping for interruption at block 19055567 23% completed, 31046803 MB accessed in 6:31 ~~~h01 1% | h02 1% | h03 1% | h04 1% | h05 0% | h06 0% | h07 0% | (...) d33 0% | d34 0% | d35 0% | parity 0% | 2-parity 0% | 3-parity 0% | 4-parity 0% | raid 59% | hash 19% | sched 2% | * misc 0% | wait time (total, less is better) ~~~ ...so clearly it's the computation that's...
-
Data rate for an array running on this system was only 1356 MB/s, with the following output: ^C%, 31045822 MB, 1356 MB/s, CPU 85%, 21:02 ETA Stopping for interruption at block 19055567 23% completed, 31046803 MB accessed in 6:31 ~~~h01 1% | h02 1% | h03 1% | h04 1% | h05 0% | h06 0% | h07 0% | (...) d33 0% | d34 0% | d35 0% | parity 0% | 2-parity 0% | 3-parity 0% | 4-parity 0% | raid 59% | ***** hash 19% | **** sched 2% | misc 0% | |____________ wait time (total, less is better) ~~~ ...so clearly...
-
Data rate for an array running on this system was only 1356 MB/s, with the following output: ^C%, 31045822 MB, 1356 MB/s, CPU 85%, 21:02 ETA Stopping for interruption at block 19055567 23% completed, 31046803 MB accessed in 6:31 h01 1% |~~~ h02 1% | h03 1% | h04 1% | h05 0% | h06 0% | h07 0% | h08 0% | h09 1% | d10 0% | d11 1% | d12 1% | d13 1% | d14 1% | d15 0% | d16 0% | d17 0% | d18 0% | d19 0% | d20 0% | d21 0% | d22 0% | d23 0% | d24 0% | d25 0% | d26 0% | d27 0% | d28 0% | d29 0% | d30 0% |...
-
I am aware of the UUID note in the FAQ, however UUID's appear to be easily obtained for ZFS volumes, so my recommendation would be to either add the ZFS volume UUID capability or to enable the ability to bypass the warning (via config) for those using volumes without UUID support. Separately, I suspected that the inability to obrain SMART data might be connected to this UUID issue, which is why both issues were brought up together. I can have the /dev/mapper/ paths in the smartctl config entries...
-
Using SnapRAID laid on top of four single-disk ZFS pools. (ZFS chosen for snapshot capability). ZFS volumes z1, z2, z3, and z4 are mounted under /z The pool directory on each volume is z z1, z2, z3 are data, z4 is parity smartctl is configured to point to the associated storage devices this is a multipath configuration, so the smartctl entries point to one of the paths to that disk Config: content /z/z1/snapraid1.content content /z/z2/snapraid2.content content /z/z3/snapraid3.content content /z/z4/snapraidp.content...
-
Using SnapRAID laid on top of four single-disk ZFS pools. (ZFS chosen for snapshot capability). ZFS volumes z1, z2, z3, and z4 are mounted under /z The pool directory on each volume is z z1, z2, z3 are data, z4 is parity smartctl is configured to point to the associated storage devices this is a multipath configuration, so the smartctl entries point to one of the paths to that disk Config: content /z/z1/snapraid1.content content /z/z2/snapraid2.content content /z/z3/snapraid3.content content /z/z4/snapraidp.content...
-
Using SnapRAID laid on top of four single-disk ZFS pools. (ZFS chosen for snapshot capability). ZFS volumes z1, z2, z3, and z4 are mounted under /z The pool directory on each volume is z z1, z2, z3 are data, z4 is parity smartctl is configured to point to the associated storage devices this is a multipath configuration, so the smartctl entries point to one of the paths to that disk Config: content /z/z1/snapraid1.content content /z/z2/snapraid2.content content /z/z3/snapraid3.content content /z/z4/snapraidp.content...
-
I know this topic goes back to 2013, and it is true that sync is very fast on a single CPU (as noted in the todo), but there is certainly some room for improvement in 'fix' operation speeds. Single drive fix throughput is reasonable at 2-3GB/s, but with modern drive capacities pushing 16TB and folks building larger arrrays and more parity disks with SnapRAID, single threaded fix operations will always be CPU limited when these larger arrays experience a disk failure. The higher the number of failed...
-
I know this topic goes back to 2013, and it is true that sync is very fast on a single CPU (as noted in the todo), but there is certainly some room for improvement in 'fix' operation speeds. Single drive fix throughput is reasonable at 2-3GB/s, but with modern drive capacities pushing 16TB and folks building larger arrrays and more parity disks with SnapRAID, single threaded fix operations will always be CPU limited when these larger arrays experience a disk failure. The higher the number of failed...
-
I noted slow snapraid fix performance on a server and some testing showed unexpectedly slow performance for snapraid sync and fix. Here are the three systems, along with cinebench R20 (AVX) single threaded results and snapraid -T results. Note that the AVX2 'check' speeds are significantly slower (nearly half) on the Xeon while it has similar single core performance (noted with the Cinebench R20 result). Memset, CRC, and hash speeds were similar across all platforms, even on the Xeon. 7900X (OC 4.6GHz):...
-
I noticed the patch go in very quickly. Thanks for adding this possibility. It will...
-
I noticed the patch go in very quickly. Thanks for adding this possibility. It will...
-
I'd like to run measurement reports of uncalibrated displays (for evaluating out-of-box...
1
