Re: [Modules] Get list of available (virtual) modules from within .modulerc

[Xavier D. <xav...@gm...>]

Le mer. 30 mars 2022 à 14:35, Bloecker, Martin <
Mar...@am...> a écrit :

> All,
>
>
>
> First of all a big thank you to Xavier and everybody on this list. It is
> good to see the community efforts to keep environment modules an active
> project.
>
>
>
> Hopefully the community can help me to tackle a problem I’m trying to
> solve:
>
> I do have a set of dynamically generated modules named (for the sake of
> abstraction) ‘vendor/software/version’. These are virtual modules that are
> defined (in vendor/software/.modulerc) by scanning (multiple) SW root
> directories for the presence of appropriate version-specific
> subdirectories. This approach works well and a list of created virtual
> modules can be output via:
>
> module avail --terse --output= software/vendor
>
> Note that these modules will only set the bare minimum environment
> required for the software. Also not all SW versions might be available on
> all platforms.
>
>
>
> Now for the actual problem:
>
> We need a second set of virtual modules named ‘localized-software/version’
> (note the different name space) with a one to one version mapping to the
> ‘vendor/software’ modules.
>
> As the ‘vendor/software/version’ are generated dynamically per platform
> the same applies to the ‘localized-software/version’ modules. In order to
> decouple the ‘vendor/software’ module implementation from the
> ‘localized-software’ module implementation as much as possible the idea is
> to use a (filtered) list of available modules (i.e. the list of
> ‘vendor/software/version’ modules) to determine which virtual
> ‘localized-software/version’ modules need to be generated.
>
>
>
> Is there a way to get a list of (currently) available modules from within
> a .modulerc file? Unfortunately the intuitive
>
>                 set sw_versions [module avail --terse --output=
> software/vendor]
>
> or
>
>                 set sw_versions [system module avail --terse --output=
> software/vendor]
>
> do not work. Note that the ‘is-avail’ function would not be useful here as
> one would need to know beforehand which potential versions will exist.
>
>
>
> Is there a way to implement what I’m trying to achieve? And a bonus
> question: In case an implementation is possible, would we need to take
> precautions against recursively processing the same .modulerc file over and
> over again? How?
>
>
>
> Thanks in advance,
>
>
>
> Martin
>

Hello Martin,

Such code could be written (using MODULES_CMD to locate modulecmd.tcl
script):

  set sw_versions [exec 2>@stdout $env(MODULES_CMD) sh avail --terse
--output= software/vendor]

But I would not recommend to have such external call in modulerc files, as
such processing could be expensive and as you mention there is a risk of
infinite processing loop.

I do not totally get the link between software/vendor and
localized-software modules, but I would suggest to extend the way the
versions of software/vendor are queried to also generate the versions for
localized-software modules. Maybe using the .modulerc file at the root of
the modulepath could help to generate all virtual modules in the same
script, using results of software/vendor creating to also help creating
localized-software modules.

Regards,
Xavier