Aaron W. LaFramboise wrote:
> Keith Marshall wrote:
>> [*] Although BZ2 is commonly better than GZ, I have seen examples
>> where GZ can be up to 20% better than BZ2, (I believe the execwrap
>> package is one such). Aaron did some testing, when we discussed this
>> previously, which showed a significant advantage for LZMA, in every
>> case he evaluated.
> Greg Chicares did most of the testing here. In fact, the best
> performance comes from 7zip either lrzip (LZMA is a pass of both), with
> lrzip slightly faster, but fairly rare, and no Windows ports.
> If we're going to switch, 7-Zip is probably the best bet for now. That
> doesn't mean we have to use its archiver features; tar+7zip is just as good.
> One important feature, present in rzip-based formats (such as lrzip) and
> to a lesser extent 7zip, is long-distance redundancy elimination. That
> is responsible for the massive advantages of lrzip and 7zip over raw lzma.
??? "massive advantage" ???
Looks like raw lzma is comparable to 7zip, both are significantly better
than rzip, and WAY better than p7zip.
So, I'd recommend .tar.lzma.
I recently released a modified version of bsdtar (that is, libarchive)
for cygwin -- with the intention of porting it to msys and native mingw.
This modified version of libarchive, in addition to supporting ISO,
.tar.gz, .tar.bz2, and all the other "normal" archive formats, supports
.tar.lzma (unpack only). The advantage of libarchive is that it does
NOT use external programs and pipes for [de]compression, unlike GNU tar
-- which is why native ports of GNU tar can't be used with
.tar.[Z|gz|bz2|lzma]: no native win32 support for "pipes" [*] OTOH,
there exists laready a native port of libarchive at the GNUWin32
project; but it doesn't have the patch for lzma (decomp) support. Yet.
I hope to make a native port of libarchive, with lzma decompression
support, available in time to be used by the new installer (and
do-it-yourself junkies will also be able to use the accompanying bsdtar).
Assuming my native libarchive+lzma is is released on time -- and maybe
even if not! -- I'd recommend raw .tar.lzma -- WHERE the tarballs
themselves are, by policy, created using --hard-dereference. (Downside:
the native port of bsdtar won't support the --hard-dereference option,
and won't support lzma *compressino*. So there's some asymmetry. Package
creators would have to use MSYS-GNU-tar + MSYS-lzma.exe, while end users
would be free to use the native bsdtar, or the new installer, or their
favorite standalone lzma.exe and tar.exe cygwin? other native ports?)
Now, --hard-dereference doesn't affect archive unpacking; it doesn't
magically turn hardlinks in a tarball into separate copies. Instead,
when the tarball is *created*, it turns hardlinks on disk into separate
copies in the tarball.
(1) on the plus side, this means that any end-users' tar-aware unpacker
can be used to unpack the file; it doesn't need to know about hardlinks
IOW, the new installer's unarchive code doesn't need to be complicated
by hardlink support -- which, if it uses libarchive, is nice because
IIRC the existing native w32 port patches for it do not provide that
support. (Sure, the MSYS link() function is implemented as a copy
operation, but that affects only msys-tar. Not any native w32 unpacker.)
(2) on the minus side, the tarballs themselves are large (unless the
compression format is sophisticated enough and has a large enough
dictionary to eliminate the redundancy).
The good news is, according to my testing, even raw lzma meets the
I'd like to avoid .7z because (a) IMO it's much more complicated to
integrate support for that format into the new installer. You can't use
the LZMA_Alone support from the LZMA SDK; you pretty much have to use
the entire SDK for full .7z support (b) the LZMA SDK is not
well-organized as a library: "use these four files for x, those 7 files
for y..." (c) the native win32 client application is GUI only IIRC, not
command line. (d) the p7zip "unix" port of the client app -- which
provides a commandline unpacker -- has no "reverse port" back to native
w32 (plus, without the Debian patches, its command line syntax is
Not a fan. Much prefer .tar.lzma, which according to Greg's tests are
equivalent to .7z in compression.
[*] please, don't be pedantic on this point. Win32 has pipes, but they
are different than unix-derived programs typically expect. native w32
ports of GNU tar don't handle compressed tar archives. Full stop.