Re: [Tapioca-devel] More on block sizes

[Richard F. <rj...@fi...>]

On Mon, 2 Jul 2001, Eric Lee Green wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hmm, seems we have three different sizes to worry about:
>
> 1. Chunk size -- if we are chunking the data into, say, 128K chunks
> 2. Record size -- we have variable sized records within those chunks.
> 3. Tape I/O size -- this may be anything from 512 bytes all the way to
>    128K+ bytes. The only real criteria is that either chunk size is a
>   multiple of this, or this is a multiple of chunk size.
>
> Things getting complicated. How to simplify?

We don't worry about Tape IO size.  We only deal with Chunk size, which is
both our IO size and the base-level archive structure size.  It is
configurable between 16k (?) and 512K (?).

The only thing about the IO size is that since a stream header cannot be
split across IO buffers, it must fit within a buffer.  If we set the
bottom end of this to 512-bytes, for example, then a file with a long
name, a resource fork, and ACLs, would not be capable of being backed up,
since it's stream header would not fit in an IO buffer.  That's why I
think a lower limit of 16k on this guy is appropriate.

The IO size get's stored in the archive header block, so we know how much
date to read/write with each IO operation ( in case we didn't figure it
out while trying to read the header block! ).

We could separate the archive block size and IO buffer size, but I don't
see any real-world advantage to doing so.  If we do this, then as you
said, we just need to ensure that either the IO size or the archive block
size is an even multiple of the other.

-- 
Richard Fish, Unix/Linux Software Engineer, rj...@fi...