#142 Parallel transfers from/to multiple disks

[Patrick Proba]

Hi Andrew,

I'm starting to like AnyBackup more and more.
One way to improve the software would be the usage of parallel file transfers. For example, if you're starting a backup process and 4 backup disks would be involved, AnyBackup will currently copy the new files (or delete old files) on these 4 disks one after another. If it would read/write from/to 2 (or more) disks paralelly, the backup time would be drastically reduced.

I guess that the majority of users of AnyBackup use a source volume with better read/write performance than just a single external backup hard drive, so during the backup the utilization of this large/fast source volume is fairly low.

If there was an option to specify how many parallel transfers would be used during such a backup, it would help anyone who connects more than one backup disk to their pc at a time.

I've set the priority to 'Low', because this might not be of help for every single user and the software is fully functional without it, but it's something which would be nice to have.

Regards,
Patrick

[Andrew Hopkinson]

I'm not against this, but it's fundamentally different than the workflow the program was designed for. If the number of parallel copies is configurable and you could have an arbitrary number of drives copying at once, how is that data aggregated and displayed effectively and in a way that scales well? How would separate workflows be handled? (i.e. say multiple backup drives need to be connected so multiple prompts would need to be shown.) Is it sufficient to do this in threads or would it require multiprocessing to get better throughput?

Basically it'd be a fairly substantial rewrite, I'd assume it'd involve moving towards an architecture with a main controller thread that talks to sqlite and the WxWidgets GUI frameworks and then several child/worker processes with quite a bit of inter-process communication in between.

I'll leave this ticket open if someone wants to take a stab at it, but it's probably not something I'll get around to doing.