#1317 An LZX decoder problem: feature or bug?

open-wont-fix
nobody
None
5
2014-08-18
2012-09-18
maxpat78
No

While experimenting with the LZX compressor found in MSCompression.dll at https://github.com/coderforlife/ms-compress, we discovered a strange behaviour in 7-Zip.

Actually, the compressor for CAB-LZX (not for CAB-WIM) is broken, so that it performs only the Huffman step, not the LZ compression: the resulting CAB, you can find attached, is about twice the size it should be.

Obviuosly, MS cabarc.exe 5.x fails (\"Failure decompressing data\") to extract such a cabinet.
7-Zip, instead, extracts it fine!!!

Was 7-zip designed to do so, or is it a bug?

Moreover, some years ago you stated that MS LZX implementation was very similar to LZMA and hard to improve.
Perhaps that it is true about compression ratio, but not about compression speed/robustness.
I tried to compress with MS cabarc 5.2 the well known canterbury corpus and found that 7-ZIP/LZMA shrinks it (better) in just 1 second or so on my old Intel Core Duo 1.66GHz, while cabarc takes A FEW MINUTES!!!
Probably, their match finder wasn\'t so well designed and improved...

Discussion

  • maxpat78

    maxpat78 - 2012-09-18

    A sample file (Huffman) cabbed.

     
  • Igor Pavlov

    Igor Pavlov - 2012-09-18

    1) It's difficult to say how LZX decoder must work, if original LZX encoder doesn't use some feature.

    2) old LZX encoder had that weakness with encoding speed.
    It can be very slow (maybe 100-1000 times slower) for some rare files. And there is such file in that corpus. It can be slow for some .pdf files also as I remember.
    It can be fixed in LZX encoder code. LZMA just stops matchfinder's loop, if it works too long.
    But they in MS probably don't care about encoder speed.
    Maybe they already have fixed that problem in new versions of LZX encoder.

     
  • Igor Pavlov

    Igor Pavlov - 2012-09-18
    • status: open --> open-wont-fix
     
  • maxpat78

    maxpat78 - 2012-09-18

    Ah, and, yes: that library provides working XPRESS compression, too... do you plan to implement WIM compression support in future 7-Zip releases?

     
  • Igor Pavlov

    Igor Pavlov - 2012-09-18

    XPRESS is not interesting for me.
    If I implement it, probaly I'll do it with LZX.
    But I have no plans for it now.

    I checked some things with WIM compression in past.
    And Microsoft WIM decoder doesn't support big dictionary for LZX.
    With small dictionary original LZX implementation is good already as I suppose.

     
  • maxpat78

    maxpat78 - 2012-10-17

    A note. We found some erroneous LZX chunks generated by that library 7-zip is able to parse/extract/test, while ImageX fails stating they are corrupt.

     

Log in to post a comment.