Incorrect warning

[Dmitry Semyonov]

We noticed that 7-Zip displays a Headers Error warning for Zip archives that seams to be correct.

The warning is triggered for Zip64 archives when reading Zip64 extended information extra field in the local file header. The warning is present in the latest 7-Zip version 18.05, 7-Zip 16.02 doesn't have such problem.

The following code is used to parse Zip64 extra field (in CPP/7zip/Archive/Zip/ZipIn.cpp):

bool CInArchive::ReadExtra(unsigned extraSize, CExtraBlock &extra,
    UInt64 &unpackSize, UInt64 &packSize, UInt64 &localOffset, UInt32 &disk)
{
...
      bool isOK = true;

      if (ZIP64_IS_32_MAX(unpackSize))
        if (size < 8) isOK = false; else { size -= 8; unpackSize = ReadUInt64(); }

      if (isOK && ZIP64_IS_32_MAX(packSize))
        if (size < 8) isOK = false; else { size -= 8; packSize = ReadUInt64(); }

      if (isOK && ZIP64_IS_32_MAX(localOffset))
        if (size < 8) isOK = false; else { size -= 8; localOffset = ReadUInt64(); }

      if (isOK && ZIP64_IS_16_MAX(disk))
        if (size < 4) isOK = false; else { size -= 4; disk = ReadUInt32(); }

      if (!isOK || size != 0)
      {
        HeadersWarning = true;
        extra.Error = true;
        extra.IsZip64_Error = true;
        Skip(size);
      }
...
}

As can be seen from the code, uncompressed and compressed sizes are expected to be present only if corresponding 32 bit values contain 0xffffffff values.

But in the APPNOTE.TXT it is stated that:

4.5.3 -Zip64 Extended Information Extra Field (0x0001):

This entry in the Local header MUST include BOTH original and compressed file size fields.

The requirement of having both fields in the local file header is actually very useful when compressing streamed data, as compressed/uncompressed sizes are not known in advance. It makes it possible to reserve space for Zip64 extra field before writing file data, and overwrite it with correct values after the data is written and the sizes are calculated. We use this trick a lot in our software, but recently we started to get reports from our users who see these warnings in 7-Zip.

[Igor Pavlov]

  The order of the fields in the zip64 extended 
  information record is fixed, but the fields MUST
  only appear if the corresponding Local or Central
  directory record field is set to 0xFFFF or 0xFFFFFFFF.

So write 0xFFFFFFFF in 32-bit field.

[Dmitry Semyonov]

Thank you for suggestion. Unfortunately we need to be compatible with some applications without Zip64 support (when the size of the data permits), so we have to place correct data in the 32bit fields if it fits there.

[Igor Pavlov]

Then you have some additional bytes to write.
You can create another extra block with some new ID.
All zip unpacking programs will skip that extra block.
But zip64 extra block will be OK in that case

[Dmitry Semyonov]

What I was trying to state is that 7-Zip seems to report a warning for a correct ZIP archive. Moreover, if the packed file size is <2^32 and at the same time unpacked file size is >2^32, the current code in CInArchive::ReadExtra will likely produce incorrect results, as it will expect only unpacked size to be present, while the ZIP spec states that both unpacked and packed sizes should be present in local header in such case.

[Igor Pavlov]

I'm sure 7-Zip works correctly, and your code probably creates incorrect archives.
Same code in info zip:

        if (eb_id == EF_PKSZ64) {

          int offset = EB_HEADSIZE;

          if (G.crec.ucsize == 0xffffffff || G.lrec.ucsize == 0xffffffff){
            G.lrec.ucsize = G.crec.ucsize = makeint64(offset + ef_buf);
            offset += sizeof(G.crec.ucsize);
          }
          if (G.crec.csize == 0xffffffff || G.lrec.csize == 0xffffffff){
            G.csize = G.lrec.csize = G.crec.csize = makeint64(offset + ef_buf);
            offset += sizeof(G.crec.csize);
          }
          if (G.crec.relative_offset_local_header == 0xffffffff){
            G.crec.relative_offset_local_header = makeint64(offset + ef_buf);
            offset += sizeof(G.crec.relative_offset_local_header);
          }
          if (G.crec.disk_number_start == 0xffff){
            G.crec.disk_number_start = (zuvl_t)makelong(offset + ef_buf);
            offset += sizeof(G.crec.disk_number_start);
          }
        }

[Igor Pavlov]

but the fields MUST
      only appear if the corresponding Local or Central
      directory record field is set to 0xFFFF or 0xFFFFFFFF

So you must write 0xFFFFFFFF there.

[Igor Pavlov]

you can try to create zip archive
where unpackSize is 32-bit only
and packSize is 0xFFFFFFFF and 64-bit in zip64 header
Then test that archive with all programs.

[Igor Pavlov]

Also you can do it so:
1) if you have 32-bit values, don't write zip64 header at all. And write some another extra block with same size
2) if you have 64-bit value in pack_size or unpack_size, then write 0xFFFFFFFF/0xFFFFFFFF and zip64 header with both 64-bit values.

[Dmitry Semyonov]

Thank you for your comments, I think you are right that both requirements in the APPNOTE need to be fulfilled together. In other words both 32bit packed_size and unpacked_size fields should be set to 2^32-1 in local header if Zip64 extended field is present. Looks like we will have to adjust our implementation, thank you once again.