|
From: Danny S. <Dan...@EM...> - 2023-04-22 17:08:52
|
Hi Xavier,
in our case we have (workload manager) feature to export/transport all environment variables from the submit to the execution host (PBS # -V). It works except for PATH which gets overwritten by something very minimal. We don’t know why (yet), but this is not our topic here. Also it’s a very dangerous feature with high potential of messing up things completely.
For module it means LOADEDMODULES u. _LMFILES_ as well as all other environment variables besides PATH are still intact. The system profile mechanism gets started (after that) and is trying to load the default modules (which doesn’t work for 4.x anymore):
module load abc
module load def
module load ghi
and so on.
The reload command is quite interesting because it would restore the exact same modules and module versions from the origin environment (when using the export feature of course). That would even improve the situation as we had with module 3.x. 😉
Of course loading the system defaults can cause conflicts, but that we had with module 3.x as well.
Thank you and Best regards,
Danny
From: Xavier Delaruelle <xav...@gm...>
Sent: 22 April 2023 07:31
To: Environment Modules usage and discussion. <mod...@li...>
Subject: Re: [Modules] "module load" compatibility
In the situation you describe, some modules are already loaded, their environment variable are set but shell alias and function vanished. This is typically the case when starting a subshell session. With Modules v4, you can "module reload"
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
Report Suspicious <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/BQNorrFsuw!1NtAYp1Rtx2EjfFl4uvPIt3W-snY2vc32OWylQR619ICJXrcURYvP5xA6-P-K8jVhvgL-8Iq0Qj_9rewUpjdt8egRvGCU_zKO9OkdGPC8kHQQghIMdBnKoTCXOxyxD4T6w$>
ZjQcmQRYFpfptBannerEnd
In the situation you describe, some modules are already loaded, their environment variable are set but shell alias and function vanished. This is typically the case when starting a subshell session.
With Modules v4, you can "module reload" to unload then load again all loaded modules.
Starting Modules v5, "module refresh" has been brought back and it is automatically triggered when initializing module in a subshell session [1]. So shell aliases and functions are still defined along their loaded module.
To answer Paul's question: loading again an already loaded module may be implemented in the future, yes.
Regards,
Xavier
[1] https://modules.readthedocs.io/en/latest/MIGRATING.html#improving-modules-initialization [modules.readthedocs.io]<https://urldefense.com/v3/__https:/modules.readthedocs.io/en/latest/MIGRATING.html*improving-modules-initialization__;Iw!!BQNorrFsuw!m-v-yG0GvgoGV87KPO8n3tI7q3Py7WYuBmbKi1UgTIkfe15S6uQJa9geKrPVkzfi2xKBHT9Qm66JTTlxqDlTClCCRzwtQo_9$>
Le ven. 21 avr. 2023 à 20:15, Danny Sternkopf <Dan...@em...<mailto:Dan...@em...>> a écrit :
Hi Paul,
All right, I meant the base vs specific module file behaviour when using switch. And we use the base version only.
Okay, there are some drawbacks. But I think in our case we really want to load the default version.
Our use case a bit bizarre. 😉
Basically what we want is to ‘semi-recover’ from a situation where only PATH is messed up (without going into details).
So we just load the system default modules again to have at least a good enough environment.
Best regards,
Danny
From: Paul Markfort <pau...@gm...<mailto:pau...@gm...>>
Sent: 21 April 2023 17:02
To: mod...@li...<mailto:mod...@li...>
Subject: Re: [Modules] "module load" compatibility
Mine was just an example (the module I was using to test is actually call base/dot - it adds ". " to the path, at the end. base is a folder, and dot is the name of a module file in that folder). If you are loading a specific module file, I don't
ZjQcmQRYFpfptBannerStart
This Message Is From an External Sender
This message came from outside your organization.
Report Suspicious <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/BQNorrFsuw!2ltERtMYvfPKBF6NwOHrzUHitsg8S3HQplZLb3Wu_8_i_X8Bm1zPtR6DCtQe3SHjp2cRqqJKnT01toJiNJRMEB1O-FAS45QBdyfsA58$>
ZjQcmQRYFpfptBannerEnd
Mine was just an example (the module I was using to test is actually call base/dot - it adds "." to the path, at the end. base is a folder, and dot is the name of a module file in that folder).
If you are loading a specific module file, I don't foresee a problem.
If base is a folder with multiple modules under it, then this could end up unloading another module under base (if the default module isn't already loaded).
On 4/21/2023 9:47 AM, Danny Sternkopf wrote:
> Hi Paul,
>
> Thank you for the advices!
>
> As we’re just using the base module for loading I tested
> ‘module switch base base’
>
> and it seems to work (module gets reloaded incl dependencies even if loaded already). This is what we want.
>
> Do you see any problem with it in comparison to the dot version ‘ module switch base/dot base/dot’ you recommended?
> Any other drawbacks?
>
> Best regards,
> Danny
>
> From: Paul Markfort <pau...@gm...<mailto:pau...@gm...>>
> Sent: 21 April 2023 16:15
> To: mod...@li...<mailto:mod...@li...>
> Subject: Re: [Modules] "module load" compatibility
>
> I suggest (if you can) that you update to: Modules Release 5. 2. 0 (2022-11-08) (has the same issue, but has many improvements and corrections). Try using "module switch" instead of module load. Note, you can only load one module with module
> ZjQcmQRYFpfptBannerStart
> This Message Is From an External Sender
> This message came from outside your organization.
> Report Suspicious <https://us-phishalarm-ewt.proofpoint.com/EWT/v1/BQNorrFsuw!2ltPJtMZlpNr756NSmphjamxzF7ZB1hr2Q-JeXQq2fj5Akb0iruOBqzqpwYmNVNmR5ZwDpgVelvBz8Eyc5ZCLiWq-JqhWgnxMfSddmU$>
> ZjQcmQRYFpfptBannerEnd
>
> I suggest (if you can) that you update to: Modules Release 5.2.0 (2022-11-08)
>
> (has the same issue, but has many improvements and corrections).
>
>
>
> Try using "module switch" instead of module load.
>
> Note, you can only load one module with module switch.
>
>
>
> There are some issues with this.
>
> If the module you are loading is not loaded, and is the same base path as another loaded module, module switch will unload that other module.
>
>
>
> A way around this is to specify the same module for modulefile1 and modulefile2 (rather than leave modulefile1 unspecified).
>
>
>
> As in:
>
> module switch base/dot base/dot
>
>
>
> However, you should see the previous post about "module load" behavior.
>
>
>
>
>
> Notes:
>
>
>
> This behavior change from the old 3.2.10 binary version to the TCL source version caused me some headaches when I first switched (some years ago).
>
>
>
> I changed some of my shell initialization module loads to module source (supported with-out full module paths in 5.2.0), as I needed them to be processed every time the shell initialization script ran (advantage: sourced modules can't be unloaded). For instance, when I want to load a module to define shell aliases (which aren't inherited by the shell).
>
>
>
>
>
>
>
> On 4/21/2023 3:54 AM, Jens-Olaf Beismann wrote:
>
>> Dear all,
>
>>
>
>> it appears that with module 3.2.10 a load of an already loaded module is executed again while it is not with version 4.5.2. https://urldefense.com/v3/__https://modules.readthedocs.io/en/latest/changes.html__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKIRttPUj$<https://urldefense.com/v3/__https:/modules.readthedocs.io/en/latest/changes.html__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKIRttPUj$>[modules[.]readthedocs[.]io]<https://urldefense.com/v3/__https:/modules.readthedocs.io/en/latest/changes.html__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKIRttPUj$%3chttps:/urldefense.com/v3/__https:/modules.readthedocs.io/en/latest/changes.html__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKIRttPUj$%3e%5bmodules%5b.%5dreadthedocs%5b.%5dio%5d>says
>
>>
>
>> When an already loaded modulefiles is asked for load again, compatibility version will refresh the shell alias definition this modulefile hold if any, whereas Modules 4 will ignore the new load order.
>
>>
>
>> Is there a way to achieve the old behavior with version 4.5.2? Our tests with the -f option have not been successful in this respect, and as per the documentation we don't want to manipulate _LMFILES_ and LOADEDMODULES explicitly.
>
>>
>
>> Thank you and best regards,
>
>>
>
>> Jens-Olaf
>
>> ---
>
>> Dr. Jens-Olaf Beismann Benchmarking Expert
>
>> High Performance Computing NEC Deutschland GmbH https://urldefense.com/v3/__https://www.nec.com/en/global/solutions/hpc/__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKKfqIEBN$<https://urldefense.com/v3/__https:/www.nec.com/en/global/solutions/hpc/__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKKfqIEBN$>[nec[.]com]<https://urldefense.com/v3/__https:/www.nec.com/en/global/solutions/hpc/__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKKfqIEBN$%3chttps:/urldefense.com/v3/__https:/www.nec.com/en/global/solutions/hpc/__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKKfqIEBN$%3e%5bnec%5b.%5dcom%5d%3e>
><https://urldefense.com/v3/__https:/www.nec.com/en/global/solutions/hpc/__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKKfqIEBN$%3chttps:/urldefense.com/v3/__https:/www.nec.com/en/global/solutions/hpc/__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKKfqIEBN$%3e%5bnec%5b.%5dcom%5d%3e>
>> Tel: +49 431 2372063 (office) +49 152 22851521 (mobile)
>
>> ---
>
>> NEC Deutschland GmbH, Fritz-Vomfelde-Straße 14, D-40547 Düsseldorf
>
>> Geschäftsführer: Yuichi Kojima
>
>> Handelsregister Düsseldorf, HRB 57941, VAT ID DE129424743
>
>>
>
>>
>
>> _______________________________________________
>
>> Modules-interest mailing list
>
>> Mod...@li...<mailto:Mod...@li...<mailto:Mod...@li...%3cmailto:Mod...@li...>>
>
>> https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$<https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$>[lists[.]sourceforge[.]net]<https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$%3chttps:/urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$%3e%5blists%5b.%5dsourceforge%5b.%5dnet%5d%3e>
><https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$%3chttps:/urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$%3e%5blists%5b.%5dsourceforge%5b.%5dnet%5d%3e>
>>
>
>
>
> --
>
> --------------------------------------------------------
>
> The views and opinions expressed above are strictly
>
> those of the author(s). The content of this message has
>
> not been reviewed nor approved by any entity whatsoever.
>
> --------------------------------------------------------
>
> Paul FM Info: https://urldefense.com/v3/__http://paulfm.com/*paulfm/__;fg!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKERnxvvu$<https://urldefense.com/v3/__http:/paulfm.com/*paulfm/__;fg!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKERnxvvu$>[paulfm[.]com]<https://urldefense.com/v3/__http:/paulfm.com/*paulfm/__;fg!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKERnxvvu$%3chttps:/urldefense.com/v3/__http:/paulfm.com/*paulfm/__;fg!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKERnxvvu$%3e%5bpaulfm%5b.%5dcom%5d%3e>
><https://urldefense.com/v3/__http:/paulfm.com/*paulfm/__;fg!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKERnxvvu$%3chttps:/urldefense.com/v3/__http:/paulfm.com/*paulfm/__;fg!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKERnxvvu$%3e%5bpaulfm%5b.%5dcom%5d%3e>
> --------------------------------------------------------
>
>
>
>
>
> _______________________________________________
>
> Modules-interest mailing list
>
> Mod...@li...<mailto:Mod...@li...<mailto:Mod...@li...%3cmailto:Mod...@li...>>
>
> https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$<https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$>[lists[.]sourceforge[.]net]<https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$%3chttps:/urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$%3e%5blists%5b.%5dsourceforge%5b.%5dnet%5d%3e>
><https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$%3chttps:/urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!iyB3C-H01tIHSv2CgndWyb7U49AY6hVdSg5RDbM9FbXSl3nd0g3Zl38BGprZaJVR3AkUJZa4fEQi0nJiKL8jB-da$%3e%5blists%5b.%5dsourceforge%5b.%5dnet%5d%3e>
>
>
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...<mailto:Mod...@li...>
> https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!n0ZuYQRLDWt3Uk0PHxsQk7aYyUYXpEfj0-EGP_2EH1cEv_klW6CPmEyqlkizHLNJgiHTEf3Yi5cO_4cBxuaQ5Wlr$<https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!n0ZuYQRLDWt3Uk0PHxsQk7aYyUYXpEfj0-EGP_2EH1cEv_klW6CPmEyqlkizHLNJgiHTEf3Yi5cO_4cBxuaQ5Wlr$>[lists[.]sourceforge[.]net]
>
--
--------------------------------------------------------
The views and opinions expressed above are strictly
those of the author(s). The content of this message has
not been reviewed nor approved by any entity whatsoever.
--------------------------------------------------------
Paul FM Info: https://urldefense.com/v3/__http://paulfm.com/*paulfm/__;fg!!BQNorrFsuw!n0ZuYQRLDWt3Uk0PHxsQk7aYyUYXpEfj0-EGP_2EH1cEv_klW6CPmEyqlkizHLNJgiHTEf3Yi5cO_4cBxu0sXzMu$<https://urldefense.com/v3/__http:/paulfm.com/*paulfm/__;fg!!BQNorrFsuw!n0ZuYQRLDWt3Uk0PHxsQk7aYyUYXpEfj0-EGP_2EH1cEv_klW6CPmEyqlkizHLNJgiHTEf3Yi5cO_4cBxu0sXzMu$>[paulfm[.]com]
--------------------------------------------------------
_______________________________________________
Modules-interest mailing list
Mod...@li...<mailto:Mod...@li...>
https://urldefense.com/v3/__https://lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!n0ZuYQRLDWt3Uk0PHxsQk7aYyUYXpEfj0-EGP_2EH1cEv_klW6CPmEyqlkizHLNJgiHTEf3Yi5cO_4cBxuaQ5Wlr$<https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!n0ZuYQRLDWt3Uk0PHxsQk7aYyUYXpEfj0-EGP_2EH1cEv_klW6CPmEyqlkizHLNJgiHTEf3Yi5cO_4cBxuaQ5Wlr$>[lists[.]sourceforge[.]net]
_______________________________________________
Modules-interest mailing list
Mod...@li...<mailto:Mod...@li...>
https://lists.sourceforge.net/lists/listinfo/modules-interest [lists.sourceforge.net]<https://urldefense.com/v3/__https:/lists.sourceforge.net/lists/listinfo/modules-interest__;!!BQNorrFsuw!m-v-yG0GvgoGV87KPO8n3tI7q3Py7WYuBmbKi1UgTIkfe15S6uQJa9geKrPVkzfi2xKBHT9Qm66JTTlxqDlTClCCR7imbY1k$>
|
