Re: [parchive-devel] Par2 Par3 comments.

[Michael N. <mic...@gm...>]

On Wed, Jun 16, 2010 at 10:46:19AM -0400, Michael Nahas wrote:
[...]
> files; even a few thousand files if the InputSize is large.  If users want
> to do more, I'd recommend they use TAR or PkZip (which enterprising
> developers might integrate into their client for ease-of-use).

I think the current par3 design (which is the same as par2 in this respect)
is not ideal.

Currently 1 slice can not contain more than 1 file, this leads to a
series of problems like 
* requireing more slices than files (and more slices are slower)
* leaving large parts of slices unused
  Think of 10 small files, if all 10 files are lost 10 parity slices are
  needed, but if all 10 files would have been in 1 slice only 1 parity slice
  would have been needed.
* 16bit RS limits the slices to 65535 thus making 32bit RS codes (which are
  slower) required for more files

If the 1 file per slice limitation is lifted these problems disappear
more than 65535 files could be stored, multiple small files could be
recovered through a single parity slice. And slow 32bit RS codes would not
be needed. (this is why iam arguing against 32bit RS codes, it seems just
the current design that benefits from them, and this design is for many
reasons not ideal)

How can the slice limit be lifted?
quite simply by using tar. And as you say "which enterprising
developers might integrate into their client for ease-of-use"
it can be done without the user having to do anything. Both the
command line and GUI apps can just implement or call tar.

now if we take this a small step further, this integration could be
made mandatory by the spec and then only a single way of storing
multiple files would be needed to be implemented by clients.
no special cases for >65535 files and users would not even need to
know that tar is run in the background to combine the files so it
also would be friendly to uneducated/lazy users. Users would just
see a par3 GUI that accepted multiple files


-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Rewriting code that is poorly written but fully understood is good.
Rewriting code that one doesnt understand is a sign that one is less smart
then the original author, trying to rewrite it will not make it better.