#15 Zip 3.0f large file tests on Win32 and Linux

[Willus]

I've tried out zip 3.0f with unzip 6.0c on both a Win32 and Linux
(x86-64) system. My goal was particularly to check large file support
(greater than 2 GB). In all cases (except for linux FIFOs), the archive
was successfully created. In most cases, it was able to be re-extracted
(with the contents verified against the original files). There were
some issues, though, which I outline below.

WIN32 (Pentium-4, Windows XP SP2)
---------------------------------
zip30f and unzip60c built correctly using "make -f win32/makefile.gcc"
with MinGW/gcc 3.4.0. I turned off unicode support because that caused
the wildcard file searching to fail.

In all cases of archives > 2 GB, unzip 6.0c on Win32 reports the
following, even when it successfully processes the archive:

error [archive.zip]: missing <XXX> bytes in zipfile
(attempting to process anyway)
error [archive.zip]: attempt to seek before beginning of zipfile
(please check that you have transferred or created the zipfile in the
appropriate BINARY mode and that you have compiled UnZip properly)

...where <XXX> is n*2^32 such that n is an integer that makes <XXX> as
close to the zip file size as possible without exceeding it, EXCEPT for
archives greater than 2 GB and less than 4 GB, in which case n is 1 and
unzip60c will not process the archive correctly. For these archives
between 2 GB and 4 GB. This appears to be an issue with unzip 6.0c.

Archive #1: 8 files, 4.6 GB zip file, largest archived file 1.7 GB
Result: SUCCESS (created/extracted/verified).

Archive #2: 59 files, 3.1 GB zip file, largest archived file 600 MB
Result: unzip 6.0c on Win32 will not process it, but when transferred
to Linux system, unzip 6.0c on linux correctly processes it.

Archive #3: 28 files, 12.5 GB zip file, largest archived file 11.8 GB
Result: SUCCESS.

Archive #4: 5 files, 3.0 GB zip file, largest archived file 1.6 GB
Result: Same as archive #2.

WIN32 SUMMARY: Unicode not working yet; large file support works except
for unzip 6.0c's inability to process archives between
2 GB and 4 GB.

LINUX (AMD-64, Opteron, CentOS 4.3)
-----------------------------------
zip30f and unzip60c built correctly using gcc 3.4.5 and
"make -f unix/Makefile". The created executable was a 64-bit
executable (x86-64).

Archive #1: 12,291 files, 15.9 GB zip file, largest archived file 2.4 GB
Result: SUCCESS.

Archive #2: 220,000 files, 6.4 GB zip file, largest archived file 1.1 GB
Result: There was one FIFO in the file set, and this caused zip 3.0f
to hang. It got to the FIFO and just sat on it, apparently
trying to archive it. After deleting the FIFO from the file
set, the archive was successfully created/processed. There
were other special files (e.g. sockets) which were ignored
by zip 3.0f, but this behavior is identical to zip 2.3.

Archive #3: 3 files, 30.1 GB zip file, largest archived file 38.2 GB
Result: SUCCESS.

LINUX SUMMARY: Large file support works; FIFOs cause hangup
(Perhaps the FIFO read/write feature should be disabled by default?)

Example of Linux FIFO hang: zip30f archive.zip /dev/initctl

[Ed Gordon]

Logged In: YES
user_id=1172496
Originator: NO

Thanks for the testing.

The UnZip 6.0c problems should be fixed in UnZip 6.0d which may be getting close to done, but no guarantees. There are known problems with UnZip 6.0c handling archives in the 2 GB to 4 GB range.

The support of Unix FIFOs was requested by a user who actually wants it to work this way. As it's implemented, you can start zipping the FIFO in one process, then feed the FIFO in another process and have the contents added to the archive. However, this creates the problem you noted and we didn't account for. It seems the best approach is to add an option to enable FIFO support, but the default would be to skip FIFOs. This is only on Unix and we are able to tell when a FIFO is encountered so this should be an easy fix that can be made to Zip 3.0g the possible release candidate. As -FI is free that looks like a good option for enabling FIFO reading.

[Ed Gordon]

Logged In: YES
user_id=1172496
Originator: NO

The Zip problems should be fixed in Zip 3.0g. The UnZip issues may be fixed in UnZip 6.0d which may be getting close to beta release.