Fab11

Hello, Sorry to come back with a reply so late. I increased parts number from 1000 to 10000 but it was still way faster to generate. I then moved to 100000 and it looks to suit better. In the worst case, the last part was generated just an hour before the end of the sync (half an hour on average) for a 7h to 9h job duration. Difficult to say if it was quicker and how much because it was a second pass anyway. So a big part of the time rsync took to copy files in the first pass was not spent again...

Hello, I will try that and let you know. You are welcome for the ideas :) Thanks for your consideration. Best regards.

Hello, Many thanks for your detailed answer! You confirmed 1000 files is not enough, it's my opinion too. I realised that while looking at the top command output frequently. Parts files number passed on the command line to fpsync were way ahead of the last generated one, meaning fpart was faster generating parts than rsync running them. I will maybe try to generate bigger parts for intermediate runs. Another question regarding jobs (thanks for the details on the usage). I was wondering what would...

Hello, I have to migrate a large amount of data to a new storage array (around 100 TB), which is mostly composed of small files (max 100 KB) located into a 7 directory deep structure. The transfert to the new storage array has to be achieved through a SMB/CIFS share. So, Samba and small files, the worst case scenario... Using rsync or robocopy would take ages. I discovered fpart and fpsync by accident and it turned out to be the magic solution according to my speed tests. It would let me copy these...