Menu
▾
▴
#413 Feature request: expose parallel build process number as variable
In linker settings, we are currently using
-fuse-ld=gold -Wl,--threads -Wl,--thread-count -Wl,60
but only for certain targets. It would be nice if, instead of hardcoding the number of threads there, we could use a variable that would expose the value of build options -> number of parallel processes for the current compiler configuration - that way it would only try to use the number of threads that codeblocks instance is configured for.
It's not really a suitable workaround to have these in the global linker settings, because then they would apply to all targets, and we want to use a different linker (or different options entirely) for some targets.
I hate to bump a feature request, but I'm wondering if this wouldn't be quite simple to implement? It's something that would be hugely beneficial to us right now; we're using LTO with multiple threads (
-flto=60on our build machine) but hard-coding this value into the project makes it unsuitable to run on other developer machines with more reasonable numbers of cores. Being able to add the compiler option as-flto$(COMPILER_THREADS)which takes it from the user's compiler config would help a lot.ok, it was easy. Sadly there is now workaround for this with some squirrel scripting...
the custom variable is called $(PARALLEL_PROCESS_COUNT)
Ah, excellent!
Tested and confirmed working, nicely done :)
Unfortunately I don't like this patch. There are already too many variables and this is starting to show in profiles. I'd rather prefer if there was a way to read this variable from scripting. Unfortunately the config manager doesn't allow access to non scripting settings. I don't know why it is done like it is.
I'm more eager to apply a patch where this variable is added to the bindings (probably a method in the compiler factory) or the GetConfigManager call is extended to allow access to various parts of the settings. This way it will be possible to do something like:
GetConfigManager("compiler").Read(_T("parallel_processes"), 1)And then use this in a scripting expanssion.
I prefer the latter version.
How would passing a variable of this sort to linker properties in a project work with scripting? Do you have an example of something similar done elsewhere?
You can always use scripting where you can use variables with the
syntax
Ahh, I see. It seems in this case you'd have to use
[[ print(GetConfigManager("compiler").Read(_T("parallel_processes"), 1)); ]]So is all that's required then to expose the parallel_processes setting of the current target's compiler? Bear in mind that a project's multiple targets can have different compilers set so it's important to use the right one.
This is not implemented yet, but with the actual implementation and a fix in the scripting binding this would be the way to go.
see the discussion below. Are there compilers, or have you a use case, where this setting has to be compiler specific? This would need a bit more work in the sdk, because at the moment this is a compiler "global" variable, for all compiler the same...
I see; I hadn't realised it was global. No, I wouldn't say having this setting per-compiler is an essential feature at this point.
In the long term, though, supporting a process limit per compiler would be helpful - especially to support "virtual" compilers, such as distcc targets or other non-local compiler calls where you might want hundreds of simultaneous processes, while keeping the local "normal" compilers at only a few, such as where you have a laptop sending distcc requests to a large compile farm.
Another example of where per-compiler process limits would be useful: at one point we had a certain project that was both hosted and built server-side. The source would be accessed over sshfs for reading and modification, but the server would build and deploy it; in this case the compiler commands were called via an ssh wrapper, so when you'd hit "build" the virtual compiler would simply send the build commands to the server over ssh. In that case the server had many more cores than the dev machines might have had, but those same dev machines would also want to build other projects locally with normal compilers without having to reconfigure the process limit every time a different project is loaded.
However, that's just a convenience issue - overall, though, just implementing this feature at all is a much higher priority - I'd rather have this value exposed now, for the global setting, than wait for the global setting to be changed to per-compiler.
i could easily make a patch with the scripting, but i am wondering why this is stored like this. Is this the same option for all compilers?
Yes, it is global for all compilers if I remember correctly. If you have the desire to make it per compiler, then go on and do the patch. This make sense to be per compiler when the compiler doesn't work well with multiple instances. This was the case with old vc++ compiler, but I don't know if this is still the case. For linux compilers I don't think it makes any difference.
ok for the first i made a patch to make the ConfigManager usable for all configurations. There is no breaking of the api or of squirrel scripts.
The syntax is like
GetConfigManager(_T("compiler")).Read(_T("parallel_processes"), 1)greetings
Can you change the patch to no give any access to the "security" configuration? I guess it is not a good idea to allow scripts to change the sandboxing rules.
here it is, but the whole security thing is just placebo, fix this would not be a easy and from my point of view useless, because scripts are "open source" and can be revised.
VBA Script is also "OpenSource" but a serious Virus source. You could name more if you like... So that argumentation does not matter.