Search Results for "chunk"

...Unlike fixed-size blocks, content-defined chunks are cut at data-dependent boundaries, which means shifts and insertions don’t invalidate the rest of the file’s delta map. The tool maintains chunk fingerprints and a manifest so it can de-duplicate across versions and even across different files that share regions of content. This approach makes it well suited for large logs, VM images, and binary assets where small edits would otherwise force full re-uploads. The implementation focuses on being robust over unreliable networks with resumable transfers and integrity checks that verify every chunk. ...

fft is a distributed file transfer tool designed to accelerate large file movement by coordinating multiple relay nodes in parallel rather than depending on a single server’s bandwidth ceiling. Its architecture splits a transfer into concurrent “workers” that fetch or push chunks across multiple paths, improving throughput on high-latency or bandwidth-constrained links. The project is implemented in Go and exposes a straightforward command-line interface so operators can stand up senders,...