#220 Possible option to deal with large sync jobs

open
nobody
None
5
2012-06-28
2012-06-28
kmistry
No

Let me start by saying I think this is a great app - the following is not a critisism - just something that I thing would be a very useful option if it's not too much trouble...

I've read a few requests/bugs where others are saying that there is insufficient memory and problems with large sync jobs (i.e. when the are lots of directories & files to sync). I have a suggestion that you may want to consider - not sure how difficult it would be I guess it depends on how the code is structured. I'm assuming that the 1st phase of running a job is to run through the source and build up a list of files to sync - and I guess this is why you need the large amount of memory. I'm assuming the 2nd stage is to then compare this against the destination and copy across files where needed. What would be nice would be to have a new option to make this work slightly differently (so users can run current method or new method). The new method would work the same but for only for 1 directory at a time. E.g. assume the source folder structure to sync is...

C:\Source
C:\Source\Sub1
C:\Source\Sub1\ABC
C:\Source\Sub1\XYZ
C:\Source\Sub2
C:\Source\Sub3

...the new method would build the file list for C:\Source only (not any of the sub folders) and then perform the sync for files in that f9older. Then it would do the same for C:\Source\Sub1, then C:\Source\Sub1\ABC, then C:\Source\XYZ, then C:\Source\Sub2 then C:\Source\Sub3

This new method would then require a lot less memory and synchronisation would start sooner (wouldn't have to wait a long time for the initial phase). The only downside that I can think of is that this new method means you can't have the "Total Progress Bar" as you don't know how far along you are whilst synchronising - you could maybe replace this with a field showing the current directory being synchronised. I'm sure those who have large sync jobs that they leave to run (say overnight) won't be too bothered about the lack of total progress - as long as it can handle large sync jobs.

Thanks for listening!

Discussion