Lasse Collin

This discussion happened via email. This will be fixed in 5.4.1. Thanks!

Yes, the "no end marker, uncompressed size is known" LZMA API should be exported. Even more importantly, there should be an API to compress without the end marker. LZMA2 does that with internal APIs but it's not exported either. Things just need to get done, I'm badly behind reviewing even the existing patches. With the current API, a very easy workaround for decompression is to use lzma_alone_decoder(). All you need to do is prefix the raw LZMA stream with a 13-byte fake header: 1 byte: LZMA properties...

Sorry for the far too long delay. Thanks for the report! I haven't looked at LZMA SDK in a long time (I haven't gotten much done around xz in general) but it's quite possible that there are some new great optimizations. I tried xz 5.2.5 and 7z 17.04 on that file. I got 16.9 s for xz and 15.3 s for 7z. ;-) My computer is old but the relative difference in speeds wasn't as huge as for you. I wonder if there is something in the code or compilers that make the difference bigger on new processors, or...

Thanks! The LZMA data has an end of stream marker. Such .7z files are valid but quite uncommon. XZ for Java 1.9 added a feature specifically to fix this issue but it requires a tiny change to Apache Commons Compress too. I had forgotten to report it to Apache Commons Compress developers earlier, sorry. I have reported it now: https://issues.apache.org/jira/browse/COMPRESS-591

Thanks! I committed a change that uses _MSVC_LANG instead of _MSC_VER. I suppose both work but using _MSVC_LANG means that the actual C++ version is checked instead of compiler version. I haven't tested the commit but perhaps you can. Thanks! https://git.tukaani.org/?p=xz.git;a=commitdiff;h=6928aac9da6ba612780b9f72ba1d6ecbe1e8b54e

With XZ_SINGLE both input and output must fully fit in RAM at the same time. Thus you need to use XZ_PREALLOC or XZ_DYNALLOC with a big file. For example, with code modified for lc=2, using 8 KiB dictionary, and reserving 512+512 bytes for input and output buffers you would need under 20 KiB of RAM with XZ_PREALLOC or XZ_DYNALLOC to decompress as big files as you wish. With lc=2, 4 KiB dictionary, and 32 bytes for output buffer you should be around 15 KiB of RAM. For input and output buffer sizes,...

On my system lld says this: lld is a generic driver. Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld (WebAssembly) instead Which is a bit confusing but sounds like that perhaps this lld shouldn't be used directly. There are messages on the web that GNU Libtool lacks direct support for LLD. My impression is that Libtool has a severe lack of resources (no active developers). ld.lld works around the lack of direct Libtool support by adding the string "GNU" in the ld.lld --version...

I don't have much clue about Docker. On my system I need to set LD=ld.lld instead of LD=lld. Otherwise I will get the same error as you did.