I read through your technical details. There's something you could do to improve how you compare the SIF between versions of files.
If a byte is inserted midway through a document, the crc's of all of the following segments would have a mismatch and would require retransmitting.
So when you find the first segment that doesn't match you could have an algorithm that tests if the segment (or the following segment) has been offset by a number of bytes. And if it has, then you can save retransferring all of the remaining segments. Just transmit the changes and offset the archived data. It would add a little overhead but potentially save a lot of time.
It might be good to evaluate how often this would happen, maybe it's so rare it's not worth doing :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I read through your technical details. There's something you could do to improve how you compare the SIF between versions of files.
If a byte is inserted midway through a document, the crc's of all of the following segments would have a mismatch and would require retransmitting.
So when you find the first segment that doesn't match you could have an algorithm that tests if the segment (or the following segment) has been offset by a number of bytes. And if it has, then you can save retransferring all of the remaining segments. Just transmit the changes and offset the archived data. It would add a little overhead but potentially save a lot of time.
It might be good to evaluate how often this would happen, maybe it's so rare it's not worth doing :)