#569 OWLMaker: PDB errors in x86 builds with Visual Studio 2022
Lately, with the Visual Studio 2022 toolset, and the debug version of OWLMaker, I have noticed that parallel builds for the x86 platform will occasionally generate the error "LNK1318: Unexpected PDB error; RPC (23)". As the documentation describes, this error is sporadic, with many potential causes and remedies. However, I have observed it on two different systems; my desktop system running Windows 10, and my laptop running Windows 11. So, I don't think it is specific to my setup.
While I haven't tested this with earlier versions of Visual Studio, I cannot remember that this has been a problem before. It seems to have appeared recently.
With that said, I have found a simple solution: Use the x64 toolset to cross-compile for the x86 platform. In testing so far, I have not seen any LNK1318 errors when using the x64 toolset. It is easy to enable as well: Just pass "x64_x86" rather than "x86" to "vcvarsall.bat" in the environment setup.
The question now is: How should we deal with this? For full flexibility, should we add another option in OWLMaker? For example, we could add another check-box "Cross-compile 32-bit targets", or we could change the current option to "Use cross-compilation" with a drop-down showing all the options available ("x86 -> x64", "x64 -> x86", "both"). Another solution is to just force the use of the x64 toolset when running on 64-bit Windows, and equally, force cross-compilation for x64 targets when running on 32-bit Windows (in which case, using the x64 toolset is of course impossible, anyway).
Let me know your thoughts!
As suggested, support for cross-compiling 32-bit targets has now been added in OWLMaker [r6858], which pretty much circumvents this issue.
Related
Commit: [r6858]
That makes sense.
Also, I don't think we should not worry too much about OWLMaker running on a 32-bit version of Windows. Microsoft has stopped providing 32bit versions in 2020, and as far as I could determine, the usage share of them is pretty low. And especially for a developer it does not make sense to still use such a machine.
Hi Ognyan,
Thank you for your feedback. It's always appreciated!
I agree with your point about not worrying too much about OWLMaker running on 32-bit Windows. Given the declining usage of 32-bit, especially among developers, it might be a good idea to consider stopping the shipment of new 32-bit versions of OWLMaker. However, for now, it isn't much of a hassle. I have pretty much fully automated the deployment now (see OWLMaker Release Script).
While my current solution in OWLMaker gives full access to all the cross-compilers available to use, it is an expert-friendly approach. A more novice-friendly solution would be to just present best practice (e.g. always use x64 toolsets if available), thereby circumventing annoying troubleshooting of problems like this.
Currently, the default for x86 targets is to use the x86 toolset. But I have a TODO comment in the code about changing the default to use the x64 cross-compiler, when available. Do you happen to know in which Visual Studio version Microsoft started shipping the x64 cross-compiler?
HI, Vidar,
With the latest release of OWLMaker - 2024.3.6.6860, I am encountering a lot of PDB related errors while trying to build the libraries. I have tried the different cross-compilation options, but it does not help.
The only way to reliably build OWLNext is to uncheck the "Build configurations in parallel" option.
Attached are screenshots and logs from two failed builds.
With the old version, 2023.12.27.6765, I had rarely encountered such errors.
Also, two small improvement suggestions for the build output:
Hi Ognyan,
Thanks for the testing and feedback!
Try turning off multiprocessing in the toolset when doing parallel builds. With it on, I get PDB errors too, unless I limit the parallelism to 2 configurations at a time. I suspect it is a memory issue. My computer has 24 GB of RAM, by the way.
With multiprocessing off, I can successfully build 16 configurations in parallel, using OWLMaker build 6860, building all configurations of OWLNext 7.0, with cross-compilation for x86. With the same settings, all configurations of OWLNext 8 (trunk) also build successfully.
Build 6765 also fails if I turn on multiprocessing while doing parallel builds. Regarding the PDB errors in particular, note that 6765 doesn't pick up linker errors in the Error List. Build 6860 fixed this issue. Regarding builds failing more recently, I suspect the Visual Studio toolset has become more demanding over time, especially in multiprocessing mode.
Last edit: Vidar Hasfjord 2024-03-11
Hi,
I am getting these errors on both my new and my old machines, and they both have 32 GB memory, so most likely it is not a memory issue.
Turning off the multiprocessing worked with the 6860.
Also, I am trying with a 64-bit release build of the latest OWLMaker sources, and it completed successfully several times (with multiprocessing checked)
On the Build Summary page, the Started and Ended times looks like are being displayed in UTC. Wouldn't it be better if they are converted to local time?
That's a bug. Thanks for reporting. I'm in UK, so I didn't notice!
I suspect that the Visual Studio toolset in multiprocessing mode tries to utilise as much memory as possible to speed up the build. I just tested a full build of OWLNext 7.0 using the older OWLMaker build 6765. With options for parallel build and multiprocessing both turned on I got many memory related errors:
That said, it could be that the PDB errors in particular are due to some other issue, e.g. race condition on some resource.
I cannot see any changes in OWLMaker build 6860 that should have any influence, though.
Yeah, I started getting the same errors with the latest build too. It may probably be affected by other factors like if the computer is being used at the same time.
Some more observations, using the latest version:
The new OWLMaker build looks quite nice, with the latest changes to the build output pages.
And you should definitely add your name to the list of authors, you have done so much work to improve it!
Hi Ognyan, thanks for the thorough testing and feedback!
I did some testing with the multiprocessing setting on my system earlier today as well. While working with many other applications open, I couldn't even do 2x parallel builds with multiprocessing on.
However, after closing all other applications and setting Windows in Focus Assist mode (don't know if it had any effect, though), the builds started succeeding. I proceeded to test 2x, 3x, 4x, 5x, until at 6x, I had an error, but I tried again, and it succeeded on the second try. I then kept going up to a successful 12x parallel build with multiprocessing!
The gains had started flatten out from 8x and up, but the 12x run did score the best time ever with all of OWLNext 8 built in 1:43. To remind you, I have a 16-core AMD Ryzen 5950X processor with 24 GB of memory.
Thanks, it is cool to see the Build Output window collecting data as the build progresses. And it has been an interesting challenge to redesign the innards to support it. As you know, your original design was intended to just show the prepared commands before executing them, and then show the result of the build afterwards. The last step is now fully redundant, and the code is more or less halfway on its way to a doc/view design, with a better separation of concerns.
Well, thanks for creating this cool build program! It has really become a dear pet project of mine to tinker with it!
I'm more than happy and proud to be part "et al", but after your invite, I have now added a tip text in the About dialog with my name on it. :-)
PS. Let me know if the latest version works well. If so, I will tag it.