When a CA is ran on Windows 2000 or lower it always results in a failure. In the log you will see the message: "Failed to invoke custom action method. Error code 0x80020008."
This appears to be happening because of the following code in SfxCA.cpp (starting at line 557 as of build 3.0.4227.0):
VARIANT vRemotingFunctionPtr;
vRemotingFunctionPtr.vt = VT_I8;
vRemotingFunctionPtr.llVal = (LONGLONG)(g_fRunningOutOfProc ? MsiRemoteInvoke : NULL);
index = 2;
hr = SafeArrayPutElement(saArgs, &index, &vRemotingFunctionPtr);
if (FAILED(hr)) goto LExit;
The problem is the assignment of the VARIANT member vt to VT_I8. The documentation at http://msdn.microsoft.com/en-us/library/ms221627(VS.85).aspx states "VT_I8 is not available in Windows Millennium Edition and earlier versions, or Windows 2000 and earlier versions." Therefore, when hr is evaluated the error message is logged and the custom action never runs.
This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 15 days (the time period specified by
the administrator of this Tracker).
This is still an issue in v3.0.5217.0. I haven't had time to debug it but I can tell you a few things:
I'm assuming the issue is probably still related to my original description however it may be a new (but similar) issue that has emerged since your code change.
Regarding your comment: "In fact, no helpful message is being logged anymore"
To clarify, is ANYTHING being logged for the CA when that happens? Normally SfxCA will write to the log some messages about extracting files and loading the CLR... do you see those? If not, that would indicate that sfxca.dll cannot be loaded by the MSI engine.
When you say "Sometimes the custom action runs just fine", does that mean it consistently works on some machines and consistently fails on others, or that the failure is intermittent on a given machine? The former might indicate a dependency issue, while the latter... I don't know.
I am still seeing the messages about extracting the files and binding to the CLR. However, the only indication that the custom action has failed is the standard message you get from the installer service: "Action ended 13:52:05: HS_IIsGetSiteInfo. Return value 3."
The testing I am doing is all on the same machine so the failure is definitely intermittent. The MSI I am using has 4 custom actions that execute in the InstallUISequence before any dialogs are shown and about 99% of the time one of these will fail. (It's not always the same one though.)
I am also noticing that every custom action that gets invoked is leaving behind an empty temp folder (like MSI2F.tmp-). This also doesn't typically happen.
I went though the trouble of trying to debug this issue (using remote debugging) on my Windows 2000 VM. (I show a message box from InvokeManagedCustomAction and attach to the process.)
The interesting thing is that, when I am debugging, all four custom actions work perfectly fine. When I'm not debugging one of them inevitably still fails. (I did use the 'Release' configuration but turned optimizations off because my breakpoints weren't being hit.)
I can still use sfxca.dll when I build it using the 'Debug' configuration as that resolves the issue. However, it is kind of annoying to have to build it and replace it every time I upgrade to a new build.
I couldn't reproduce the intermittent failure you describe. I ran both debug and release builds of managed custom actions on a Windows 2000 Server SP4 VM numerous times with no failures. If you're able to pinpoint the problem please reactivate this bug.
I'm leaving this bug resolved as "Fixed" because the original problem in the bug description is fixed.
This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 21 days (the time period specified by
the administrator of this Tracker).