#1284 Unable to generate a solid compressed installer when size of installed files exceeds 2 GiB

[f0rt]

The bug was originally reported on the Debian bug tracking system: https://bugs.debian.org/1009846

All current versions of makensis (which is part of the nsis package) crash
when the total size of the installed files exceeds 2 GiB and compression
option /SOLID is set. I tested both the nsis package which is part of
Debian bullseye and a newer locally built version.

The crash is caused by an 32 bit integer overflow, at least in Source/mmap.cpp.
I observer SIGBUS, SIGSEGV and mmap related error messages, depending on the
files which were to be installed.

The bug can be avoided by removing /SOLID, so instead of whole file compression
only the single installed files get compressed, but that results in a larger
installer.

Fixing the bug would require lots of code changes, mainly replacing "int"
by "unsigned int" (which would have a limit at 4 GiB) or "size_t".

A check for integer overflow and aborting with an reasonable error message
would be easier to implement.

I was able to reproduce it with the latest makensis built from the sources of the subversion repository ( revision 7356).

SetCompressor /SOLID lzma

Name minimal

Section
  File big_1.bin
  File big_2.bin
SectionEnd

I created two 1.5 GiB files with random data to prevent any compression benefits.
Thereafter I tried to generate an installer via makensis and the above script.

dd if=/dev/random of=big_2.bin bs=1M count=1536
dd if=/dev/random of=big_2.bin bs=1M count=1536
makensis minimal.nsi

Processing config: /etc/nsisconf.nsh
Processing script file: "minimal.nsi" (UTF8)

Internal compiler error #12345: error mmapping file (1610612744, 33554432) is out of range.

Note: you may have one or two (large) stale temporary file(s) left in your temporary directory (Generally this only happens on Windows 9x).

[Anders]

We are limited to 2gb of compressed data because we use the top bit. The nsisbi project has removed this limit.

Stopping with an error is acceptable here, crashing is not. It does not check the size of the uncompressed files, it hopes they compress enough to fit in 2gb.

[Jason]

Just to clarify, nsisbi still has a limit, it's 64 bit minus the top bit (I just widened the existing 32 bit integer).

Single source files were limited to 2GB until 7208-1, when I made those 64bit minus the top bit as well (you can still choose the 32 bit limit with a scons command line switch).

(Edited because I didn't read the original post properly, oops).

[f0rt]

Thanks for the feedback.

I read the latest changelog of nsisbi. It states "Solid compression is not supported with big installers". So it looks to me that nsisbi would not help in case of solid compression.

[Jason]

Oops. I just realized I had been using the wrong wording all these years. It should be: "Solid compression is not supported with external file installers". I actually fixed solid compression on all-in-one installers in 2019 (the 7069-1 release).