|
From: Gsell, A. <ach...@ps...> - 2025-10-16 13:58:06
|
Dear Xavier Since our GitLab server will be shut down at the end of November, I have moved Pmodules to GitHub. The new URL is https://github.com/Pmodules/Pmodules Documentation is available at: https://pmodules.github.io/ I wrote a short document summarising the required features we would need in Modules to replace Pmodules with Modules+site specific procedures/configuration: https://github.com/Pmodules/Pmodules/wiki/Differences-between-Pmodules-and-Modules Comments/suggestions how we can proceed are very welcome! Best regards, Achim On 13 Sep 2025, at 15:16, Xavier Delaruelle <xav...@gm...> wrote: Dear Achim, Thanks for these informations. I am following the development of Pmodules since several years through the PSI gitlab. That's great if you would like to contribute here. Welcome :-) Best regards, Xavier Le ven. 12 sept. 2025 à 11:39, Gsell, Achim <ach...@ps...> a écrit : Dear Xavier, thanks a lot for the answer and solution! A little background on why I'm experimenting with this. At the Paul Scherrer Institute (PSI), we use Pmodules. Pmodules is essentially a wrapper for Environment Modules 3.2.10 written in Bash. This wrapper implements additional features not available in the old implementation of the Environment Modules. Some of them are now available in the current version, or there is similar functionality available. I started developing Pmodules at a time when the development of the Environment Modules was not progressing. This changed and some of the features in Pmodules - like hierarchical modules - are now available in the Environment Modules. Other features like overlays, Lmod support, availability of modules depending on hostname and/or OS/version are not available. Since developing and supporting a special version that is only used at PSI is lot of work, we are considering whether some of the Pmodule for PSI essential extensions could be incorporated into the Tcl Environment Modules. I would like to contribute to the project and have a significant part of my working time available for the implementation of these features. Best regards Achim On 12 Sep 2025, at 07:39, Xavier Delaruelle <xav...@gm...> wrote: Dear Achim, First of all, I made a mistake in my analysis yesterday: defining module-addpath procedure within the modulefile works just fine. The issue you face comes from the Tcl evaluation context. When defined in the siteconfig.tcl, the module-addpath procedure is evaluated in the main execution context, not the one specific to modulefile evaluation. Thus, modulefile commands like prepend-path called from the main context are not the correct one adapted to the current evaluation mode ("edit-path-sc" for scan mode or "prepend-path-un" for unload mode). To get the expected behavior, the prepend-path command should be run from current modulefile evaluation context. It could be achieved by changing the module-addpath procedure the following way: proc module-addpath {path} { # get name of current module Tcl interp set itrp [getCurrentModfileInterpName] interp eval $itrp prepend-path MODULEPATH $path } I need to update the documentation to explain that along the siteconfig.tcl modulefile_extra_cmds documentation. Best regards, Xavier Le jeu. 11 sept. 2025 à 06:14, Xavier Delaruelle <xav...@gm...> a écrit : Dear Achim, Thanks for your message. I can reproduce the issue you describe on my side. Even when defining the module-addpath procedure within the modulefile. From a first quick analysis, it does not feel that this is the indented behavior. I will need to investigate further down to find what is going on. Best regards, Xavier Le mer. 10 sept. 2025 à 10:57, Gsell, Achim <ach...@ps...> a écrit : Dear All, I'm currently playing with a hierarchical module structure, which is now supported in version 5.6. If I add a new directory to MODULEPATH in a modulefile itself, everything works fine. But as soon as I move the statement prepend-path MODULEPATH $something to a procedure inside my siteconfig.tcl file I run into two issues. First: The path $something is not removed from MODULEPATH while unloading the module. Second: module spider —indepth doesn’t show the modules in $something. Example: I have the following (simplified) modulefile for gcc/15.2.0: $ cat /afs/psi.ch/group/HPCE/Modules/Programming/modulefiles/gcc/15.2.0 #%Module module-help "GCC compiler collection" module-addpath {/afs/psi.ch/group/HPCE/Modules/Compiler/modulefiles/gcc/15.2.0} In my siteconfig.tcl the procedure module-addpath is defined as: proc module-addpath { path } { prepend-path MODULEPATH $path } I have the following modulefiles: [gsell@merlin-c-002 ~]$ find /afs/psi.ch/group/HPCE/Modules/*/modulefiles -type f /afs/psi.ch/group/HPCE/Modules/Compiler/modulefiles/gcc/15.2.0/OpenBLAS/0.3.30 /afs/psi.ch/group/HPCE/Modules/Programming/modulefiles/gcc/15.2.0 and MODULEPATH is set to: [gsell@merlin-c-002 ~]$ echo $MODULEPATH /afs/psi.ch/group/HPCE/Modules/Programming/modulefiles The spider command should show the OpenBLAS module, but it doesn’t: [gsell@merlin-c-002 ~]$ module spider --indepth ------------ /afs/psi.ch/group/HPCE/Modules/Programming/modulefiles ------------- gcc/15.2.0 Key: modulepath After loading the gcc module, the OpenBLAS module is available, this is correct: [gsell@merlin-c-002 ~]$ module load gcc/15.2.0 [gsell@merlin-c-002 ~]$ module avail -------- /afs/psi.ch/group/HPCE/Modules/Compiler/modulefiles/gcc/15.2.0 --------- OpenBLAS/0.3.30 ------------ /afs/psi.ch/group/HPCE/Modules/Programming/modulefiles ------------- gcc/15.2.0 Key: loaded modulepath But after unloading the gcc module, the OpenBLAS module is still available: [gsell@merlin-c-002 ~]$ module unload gcc/15.2.0 [gsell@merlin-c-002 ~]$ module avail -------- /afs/psi.ch/group/HPCE/Modules/Compiler/modulefiles/gcc/15.2.0 --------- OpenBLAS/0.3.30 ------------ /afs/psi.ch/group/HPCE/Modules/Programming/modulefiles ------------- gcc/15.2.0 Key: modulepath and the added directory is still in MODULEPATH: [gsell@merlin-c-002 ~]$ echo $MODULEPATH /afs/psi.ch/group/HPCE/Modules/Compiler/modulefiles/gcc/15.2.0:/afs/psi.ch/group/HPCE/Modules/Programming/modulefiles If I replace in the gcc/15.2.0 modulefile module-addpath {/afs/psi.ch/group/HPCE/Modules/Compiler/modulefiles/gcc/15.2.0} with prepend-path MODULEPATH {/afs/psi.ch/group/HPCE/Modules/Compiler/modulefiles/gcc/15.2.0} everything works as expected. I’m not sure whether this behaviour is intended or a bug. If it is intended, how can I add something to MODULEPATH in a procedure? Best regards, Achim _______________________________________________ Modules-interest mailing list Mod...@li... https://lists.sourceforge.net/lists/listinfo/modules-interest _______________________________________________ Modules-interest mailing list Mod...@li... https://lists.sourceforge.net/lists/listinfo/modules-interest _______________________________________________ Modules-interest mailing list Mod...@li... https://lists.sourceforge.net/lists/listinfo/modules-interest _______________________________________________ Modules-interest mailing list Mod...@li... https://lists.sourceforge.net/lists/listinfo/modules-interest |
