Some versions of scons don't know that the MS 64 bit assembler is called ml64, and just tries to use ml instead which scons gives a warning about.
So I made a 2 line patch that explicitly tells it that it's ml64. VS2008 and VS2019 have the same issue (these are the only versions I have right now, I do have VS2005 express lying around somewhere, I'll check that too).
So VS2005 pro is the opposite, it fails with this patch and standard nsis compiles fine. So I will have to find a way to change the assember in the sconscript for the system plugin.
Turns out I have to install the updates for VS2005, then my patch works as expected. I got an unrelated error as well before installing the updates (the original release has a few bugs in it :( ).
Is this something that should be fixed upstream in SCons instead?
Yes, I believe so. I'm not the only one that found this issue. (https://scons-users.scons.narkive.com/g4FXoAEP/scons-uses-wrong-visual-studio-assembler-for-64-bit).
I could try the latest scons and see if it's fixed yet.
[edit] Tried it with scons 4.0.1 (latest as of this post), and it still fails. I guess it's not really an issue because 64 bit nsis still isn't finalized yet.
Last edit: Jason 2020-09-12