modules-interest — Discussion forum for modules usage.

[Modules] Set a module "family provider" as a prerequisite?

[Adrien C. - G. E. <adr...@eo...>]

Hi,

I have a module with à prereq "cuda-toolkit" or "nvhpc".

So here is my prereq line:

%Module
prereq cuda-toolkit nvhpc

But, is it possible to declare my modules to be a sort of "cuda provider"?

So i juste have to declare "prereq cuda" or "prereq --provider cuda" in my module?

In fact, it's not just about cuda. I have some python modules too: intelpython, python3 and anaconda.

I would like to set them as "python3.10" providers.

I hope my ask in clear, if not please tell me.

Best,
Adrien

[Modules] MoGui

[Xavier D. <xav...@gm...>]

Hello All,

We are pleased to announce the first release of `MoGui`, the Graphical
User Interface for Modules. This simple application helps users to
manage their module environment in a graphical desktop environment.

https://github.com/cea-hpc/mogui

This first release (0.2) is available from PyPi
(https://pypi.org/project/modules-gui/). This app is still in beta
stage.

Bug report and contributions are highly welcomed

Regards,
Xavier

Re: [Modules] Add completetion for "ml" command

[Xavier D. <xav...@gm...>]

Hello Laurent,

I agree with you, it may be preferable to keep completion for "ml"
simple. Maybe add completion for long option when "--" is entered.

Best regards,
Xavier

Le mer. 21 févr. 2024 à 13:04, Laurent BESSON
<lau...@ea...> a écrit :
>
> Hello,
>
> I have tried to add all possible completion from "module" command to "ml" command but, at the end, I found it confusing:
>
> Trying to hit "tab" after:
> >>>> ml -
>
> will list all already loaded modules but also all possible options of module command (like "--version"). So the display is a bit messy with a mix of loaded modules and options. I think we are losing the simplicity of the new "ml" command whose first intention is to list/load/unload.
>
> Same when trying to complete "ml " with either the available modules and all the commands of "module".
>
> However, as suggested, I'll fire a new request under GitHub with essential completion for ml
>
> BR,
> Laurent.
>
>
> -----Message d'origine-----
> De : Xavier Delaruelle <xav...@gm...>
> Envoyé : mardi 20 février 2024 18:53
> À : Environment Modules usage and discussion. <mod...@li...>
> Objet : Re: [Modules] Add completetion for "ml" command
>
> Hello Laurent,
>
> It would be very interesting to have ml completion added for tcsh shell. Would you like to contribute and create a pull request for that?
>
> init/tcsh_completion.in file should be updated with a "complete ml"
> definition. From my understanding, this definition should equal the content of "complete module" + definition you propose.
>
> Best regards,
> Xavier
>
> Le mar. 20 févr. 2024 à 16:57, Laurent BESSON <lau...@ea...> a écrit :
> >
> > Hello,
> >
> >
> >
> > Would it be possible to add shell completion for the almost-new « ml » command.
> >
> > On my side, using tcsh as my loggin shell, I have defined the following :
> >
> >
> >
> >   complete ml 'c/-/`_module_loaded`/' \
> >
> >               'p/1/`_module_not_yet_loaded; echo "-"`/'
> >
> >
> >
> > Which is working quite well.
> >
> >
> >
> > Best regards,
> >
> > Laurent.

[Modules] Modules on Mastodon

[Xavier D. <xav...@gm...>]

Hello,

For those who prefer Mastodon over X (Twitter), I have created the
@Env...@ma...cial account.

I will relay news of the project through this profile (in addition to
this mailing-list and Twitter).

Come follow me at https://mast.hpc.social/@EnvModules

Regards,
Xavier

Re: [Modules] Add completetion for "ml" command

[Laurent B. <lau...@ea...>]

Hello,

I have tried to add all possible completion from "module" command to "ml" command but, at the end, I found it confusing:

Trying to hit "tab" after:
>>>> ml -

will list all already loaded modules but also all possible options of module command (like "--version"). So the display is a bit messy with a mix of loaded modules and options. I think we are losing the simplicity of the new "ml" command whose first intention is to list/load/unload.

Same when trying to complete "ml " with either the available modules and all the commands of "module". 

However, as suggested, I'll fire a new request under GitHub with essential completion for ml 

BR,
Laurent.


-----Message d'origine-----
De : Xavier Delaruelle <xav...@gm...> 
Envoyé : mardi 20 février 2024 18:53
À : Environment Modules usage and discussion. <mod...@li...>
Objet : Re: [Modules] Add completetion for "ml" command

Hello Laurent,

It would be very interesting to have ml completion added for tcsh shell. Would you like to contribute and create a pull request for that?

init/tcsh_completion.in file should be updated with a "complete ml"
definition. From my understanding, this definition should equal the content of "complete module" + definition you propose.

Best regards,
Xavier

Le mar. 20 févr. 2024 à 16:57, Laurent BESSON <lau...@ea...> a écrit :
>
> Hello,
>
>
>
> Would it be possible to add shell completion for the almost-new « ml » command.
>
> On my side, using tcsh as my loggin shell, I have defined the following :
>
>
>
>   complete ml 'c/-/`_module_loaded`/' \
>
>               'p/1/`_module_not_yet_loaded; echo "-"`/'
>
>
>
> Which is working quite well.
>
>
>
> Best regards,
>
> Laurent.
>
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest


_______________________________________________
Modules-interest mailing list
Mod...@li...
https://lists.sourceforge.net/lists/listinfo/modules-interest

Re: [Modules] Modules & CPAN

[Harlan S. <ha...@pf...>]

On 2/20/2024 8:38 PM, Sternberg, Michael G. via Modules-interest wrote:
> You can certainly get by with a single perl binary by putting the 
> specific version of openssl right under its nose, though a script wrapper:
> 
> ----------------------------
> #!/bin/sh
> LD_LIBRARY_PATH=/path/to/openssl/lib${LD_LIBRARY_PATH:+":$LD_LIBRARY_PATH"} \
>      exec /path/to/bin/perl \
> ${1+"$@"}
> ----------------------------
> 
> Place this script "earlier" in your PATH than your perl binary.
> 
> If you happen to control that perl binary yourself (as opposed to e.g. 
> your system's package manager), you could also move it to, say, 
> ../libexec/ and place the script in its stead. In that case, you could 
> write  "… exec $(dirname $0)/../libexec/perl …" to locate it.

Thanks!

And:

- I'd greatly prefer to avoid LD_LIBRARY_PATH, as relying on this has
   always resulted in intricate debugging problems and drama.  It also
   requires education and significant attention on the part of the
   sysadmin/releng/build teams.
- Is there a non-clunky solution to the problem of needing to compile
   N instances of perl for each of the N versions of openssl that are
   available?

My first point above is still a major concern for me.

The 2nd point is independent of the first point.  Doing N instances of 
perl+openssl may be what has to happen.  And I'm wondering what other 
toolchains will need to have their own collections:

- perl-$PERLVER--noopenssl
- perl-$PERLVER--openssl-$OPENSSLVER

H
--
> With best regards,
> -- 
> Michael Sternberg, Ph.D.
> Principal Scientific Computing Administrator
> Center for Nanoscale Materials
> Argonne National Laboratory
> 
> 
> 
>> On Feb 20, 2024, at 20:56, Harlan Stenn <ha...@pf...> wrote:…There 
>> are some CPAN modules that work with, for example, OpenSSL.  These 
>> CPAN modules need the right/appropriate OpenSSL libraries at runtime.
>>
>> CPAN modules apparently Do Not Want to be statically linked.  I think 
>> the best solution is to find a way to install CPAN modules that have 
>> been statically linked to there appropriate (at build time) libraries.
>>
>> I really don't want to use LD_LIBRARY_PATH to solve this problem, 
>> because in an environment where modules is in use, one might be using 
>> several different sets of tools, and the openssl being used by perl and 
>> the perl modules might be different from the version of openssl that is 
>> used by some other set of utilities that are needed for the current 
>> overall operating environment.
>>
>> I am trying Really Hard to not have to build a separate version of perl 
>> (and the CPAN modules) for each version of OpenSSL.
>>
>> I figure there are more "flavors" involved than just OpenSSL, so the 
>> problem above will only get worse if I have to build an entire toolchain 
>> suite for each "flavor".
> 
> 
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest

Re: [Modules] Modules & CPAN

[Sternberg, M. G. <ste...@an...>]

You can certainly get by with a single perl binary by putting the specific version of openssl right under its nose, though a script wrapper:

----------------------------
#!/bin/sh
LD_LIBRARY_PATH=/path/to/openssl/lib${LD_LIBRARY_PATH:+":$LD_LIBRARY_PATH"} \
    exec /path/to/bin/perl \
${1+"$@"}
----------------------------

Place this script "earlier" in your PATH than your perl binary.

If you happen to control that perl binary yourself (as opposed to e.g. your system's package manager), you could also move it to, say, ../libexec/ and place the script in its stead. In that case, you could write  "… exec $(dirname $0)/../libexec/perl …" to locate it.


With best regards,
--
Michael Sternberg, Ph.D.
Principal Scientific Computing Administrator
Center for Nanoscale Materials
Argonne National Laboratory



On Feb 20, 2024, at 20:56, Harlan Stenn <ha...@pf...> wrote:…There are some CPAN modules that work with, for example, OpenSSL.  These
CPAN modules need the right/appropriate OpenSSL libraries at runtime.

CPAN modules apparently Do Not Want to be statically linked.  I think
the best solution is to find a way to install CPAN modules that have
been statically linked to there appropriate (at build time) libraries.

I really don't want to use LD_LIBRARY_PATH to solve this problem,
because in an environment where modules is in use, one might be using
several different sets of tools, and the openssl being used by perl and
the perl modules might be different from the version of openssl that is
used by some other set of utilities that are needed for the current
overall operating environment.

I am trying Really Hard to not have to build a separate version of perl
(and the CPAN modules) for each version of OpenSSL.

I figure there are more "flavors" involved than just OpenSSL, so the
problem above will only get worse if I have to build an entire toolchain
suite for each "flavor".

[Modules] Modules & CPAN

[Harlan S. <ha...@pf...>]

I haven't had to deal with this in a long time, and I'm hoping somebody 
here can save me a lot of time to get a working answer.

There are some CPAN modules that work with, for example, OpenSSL.  These 
CPAN modules need the right/appropriate OpenSSL libraries at runtime.

CPAN modules apparently Do Not Want to be statically linked.  I think 
the best solution is to find a way to install CPAN modules that have 
been statically linked to there appropriate (at build time) libraries.

I really don't want to use LD_LIBRARY_PATH to solve this problem, 
because in an environment where modules is in use, one might be using 
several different sets of tools, and the openssl being used by perl and 
the perl modules might be different from the version of openssl that is 
used by some other set of utilities that are needed for the current 
overall operating environment.

I am trying Really Hard to not have to build a separate version of perl 
(and the CPAN modules) for each version of OpenSSL.

I figure there are more "flavors" involved than just OpenSSL, so the 
problem above will only get worse if I have to build an entire toolchain 
suite for each "flavor".

I'd bet that some folks here have had to deal with this situation and 
might have answers for me.

Thanks...

H

Re: [Modules] Add completetion for "ml" command

[Xavier D. <xav...@gm...>]

Hello Laurent,

It would be very interesting to have ml completion added for tcsh
shell. Would you like to contribute and create a pull request for
that?

init/tcsh_completion.in file should be updated with a "complete ml"
definition. From my understanding, this definition should equal the
content of "complete module" + definition you propose.

Best regards,
Xavier

Le mar. 20 févr. 2024 à 16:57, Laurent BESSON
<lau...@ea...> a écrit :
>
> Hello,
>
>
>
> Would it be possible to add shell completion for the almost-new « ml » command.
>
> On my side, using tcsh as my loggin shell, I have defined the following :
>
>
>
>   complete ml 'c/-/`_module_loaded`/' \
>
>               'p/1/`_module_not_yet_loaded; echo "-"`/'
>
>
>
> Which is working quite well.
>
>
>
> Best regards,
>
> Laurent.
>
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest

[Modules] Add completetion for "ml" command

[Laurent B. <lau...@ea...>]

Hello,

Would it be possible to add shell completion for the almost-new < ml > command.
On my side, using tcsh as my loggin shell, I have defined the following :

  complete ml 'c/-/`_module_loaded`/' \
              'p/1/`_module_not_yet_loaded; echo "-"`/'

Which is working quite well.

Best regards,
Laurent.

[Modules] Modules 5.4.0 released

[Xavier D. <xav...@gm...>]

Hello All,

Modules 5.4 is here! As always, new version means many new features:
Enhancing extra specifiers, Purging sticky modules, Specific
modulepath labels, Unique module name loaded, Cache sourced files,
Abort on error, Improve error reporting, New options for source-sh
modulefile command.

Details on the major changes included in this new release are provided
in the MIGRATING document. The list of all changes made in this new
version are described in the NEWS document.

 http://modules.readthedocs.io/en/stable/MIGRATING.html
 http://modules.readthedocs.io/en/stable/NEWS.html

The tarball of this new version can be downloaded at:

 http://downloads.sourceforge.net/modules/modules-5.4.0.tar.gz

The zipball to install this new version on Windows platform can be
downloaded at:

 http://downloads.sourceforge.net/modules/modules-5.4.0-win.zip

If you encounter any issue, please let us know by creating a ticket on
the project bug tracker at:

 https://github.com/cea-hpc/modules/issues

Many thanks to Laurent Chardon and Jérémy Déchard for their
contribution to this release. Many thanks also all of you who improve
Modules by reporting bugs or sharing new ideas.

Best regards,
Xavier

Re: [Modules] Beginner Installation Help

[Xavier D. <xav...@gm...>]

Hello,

If only the python tests fail like that, I assume your installation is
ok. The errors seem to be due to the use of a recent Python version. I
will check running tests against Python 2.12. Maybe I have some test
scripts to fix.

For the configuration of the Modules tool, you can refer to the
Configuration section of the installation guide:
https://modules.readthedocs.io/en/latest/INSTALL.html#configuration

To create modulefiles to use with module commands to configure access
to specific software or just tune your shell session, you  just need
to know the environment element to setup and then create files with
modulefile commands to define these elements:
* setenv: define environment variable
* append-path/prepend-path: add path element to a path-like variable
* set-alias: define shell alias
* set-function: define shell function
* complete: define completion

All modulefile commands are described in modulefile man page:
https://modules.readthedocs.io/en/latest/modulefile.html#modules-specific-tcl-commands

Regards,
Xavier

Le jeu. 15 févr. 2024 à 16:53, Ameya Vikram Singh
<ame...@gm...> a écrit :
>
> Hello,
>
> I am trying to build and install the environment modules from source.
> Though the build is successful, the testinstall is currently failing.
>
> Here's brief snippet I have captured around the failing test:
> Running /home/ameya/src/environment-modules/modules/testsuite/install.00-init/080-args.exp
> ...
> FAIL: append-path,FOO,\$foo:source,/home/ameya/src/environment-modules/modules/testsuite/modulefiles.2/putsvar
> (python)
> ERR[77]: '<string>:3: SyntaxWarning: invalid escape sequence '\$'
> :/path/to/dir1:\$foo
> '#>
> EXP[25]: '^:/path/to/dir1:\\\$foo
> $'#>
> FAIL: append-path,FOO,\$foo:source,/home/ameya/src/environment-modules/modules/testsuite/modulefiles.2/putsvar
> (python)
> ERR[77]: '<string>:3: SyntaxWarning: invalid escape sequence '\$'
> :/path/to/dir1:\$foo
> '#>
> EXP[25]: '^:/path/to/dir1:\\\$foo
> $'#>
>         Unset temp variables: err_locatemod premsg postmsg usage_msg
> err_locatemod2 err_locatemod3 err_locatemod4 err_locatemod5
> err_locatemod6 err_locatemod7 foo err_emptydir err_emptymod modfile
> bar
>
> I am currently using Fedora 39 as the base system.
> Also, I don't have admin rights to install packages, for which I need
> to install modules locally using the following configuration commands:
>
> # Configure
> ./configure --prefix=$HOME/tools/modules
> # Build
> make
> # Do Self test (this one works)
> make test
> # Install
> make install
> # Source Installation
> source $HOME/tools/modules/init/bash
>
> # Test The Installation (This one is currently failing with the error
> captured above.)
> make testinstall
>
> Also, is there any guide I can refer to on configuring custom tools
> setup configuration?
> I am not sure which section of the document should I refer to for a
> custom application configuration on modules?
>
> --
> Regards,
> Ameya

[Modules] Beginner Installation Help

[Ameya V. S. <ame...@gm...>]

Hello,

I am trying to build and install the environment modules from source.
Though the build is successful, the testinstall is currently failing.

Here's brief snippet I have captured around the failing test:
Running /home/ameya/src/environment-modules/modules/testsuite/install.00-init/080-args.exp
...
FAIL: append-path,FOO,\$foo:source,/home/ameya/src/environment-modules/modules/testsuite/modulefiles.2/putsvar
(python)
ERR[77]: '<string>:3: SyntaxWarning: invalid escape sequence '\$'
:/path/to/dir1:\$foo
'#>
EXP[25]: '^:/path/to/dir1:\\\$foo
$'#>
FAIL: append-path,FOO,\$foo:source,/home/ameya/src/environment-modules/modules/testsuite/modulefiles.2/putsvar
(python)
ERR[77]: '<string>:3: SyntaxWarning: invalid escape sequence '\$'
:/path/to/dir1:\$foo
'#>
EXP[25]: '^:/path/to/dir1:\\\$foo
$'#>
        Unset temp variables: err_locatemod premsg postmsg usage_msg
err_locatemod2 err_locatemod3 err_locatemod4 err_locatemod5
err_locatemod6 err_locatemod7 foo err_emptydir err_emptymod modfile
bar

I am currently using Fedora 39 as the base system.
Also, I don't have admin rights to install packages, for which I need
to install modules locally using the following configuration commands:

# Configure
./configure --prefix=$HOME/tools/modules
# Build
make
# Do Self test (this one works)
make test
# Install
make install
# Source Installation
source $HOME/tools/modules/init/bash

# Test The Installation (This one is currently failing with the error
captured above.)
make testinstall

Also, is there any guide I can refer to on configuring custom tools
setup configuration?
I am not sure which section of the document should I refer to for a
custom application configuration on modules?

-- 
Regards,
Ameya

[Modules] New options for source-sh modulefile command

[Xavier D. <xav...@gm...>]

Dear All,

New options for `source-sh` modulefile command are introduced for next
Modules version (5.4):

* `bash-eval` shell mode to get environment changes from script output.
* `--ignore` option to skip some kind of environment changes.

See details and examples at:
https://modules.readthedocs.io/en/latest/MIGRATING.html#new-options-for-source-sh-modulefile-command

Regards,
Xavier

[Modules] Improve error reporting

[Xavier D. <xav...@gm...>]

Dear All,

Error reporting has been tweaked for next version (5.4). When dealing
with module dependencies, errors are now reported where they occur.
Specific errors for switch sub-command and module conficts have been
improved.

See details and example at:

https://modules.readthedocs.io/en/latest/MIGRATING.html#improve-error-reporting

Regards,
Xavier

Re: [Modules] Question: Modules v3.2.10 logging

[Lex G. <lex...@mo...>]

Hi Xavier,

Thanks for your help. We’ve ultimately gone for a different option where we hack the modules bash initialisation scripts on all of our nodes, which is just slightly less awkward for us then modifying all our existing module files. This is only short-term anyway, we’ll have an upgraded system soon enough.

Thanks again though, good to know the `trace` approach was futile!


Cheers,
Lex.

> On 25 Jan 2024, at 6:11 pm, Xavier Delaruelle <xav...@gm...> wrote:
> 
> Hi Lex,
> 
> `trace` command requires to be used in modulecmd.tcl customization
> script (siteconfig.tcl). This is not available on Modules 3.
> 
> I recommend setting up an `exec` command in every modulefiles you want
> to log activity.
> 
> if { [module-info mode] in {load unload} } {
>    exec logger -t module "$env(USER) [module-info mode] [module-info name]"
> }
> 
> The above code snippet should send a log message toward your log
> system when module is loaded or unloaded.
> 
> Regards,
> Xavier
> 
>> Le jeu. 25 janv. 2024 à 07:26, Lex Gallon via Modules-interest
>> <mod...@li...> a écrit :
>> 
>> Hi all,
>> 
>> Firstly, apologies if this is a silly question. I've never used TCL outside of module files before.
>> 
>> I'm on a shared system where we are stuck with Modules v3.2.10 (TCL 8.5). I've been trying to enable logging so we have a record of which modules are being loaded most often, but I've been struggling to configure it properly.
>> 
>> Under `man module`, I found that I can add a `~/.modulerc` which gets executed on any module commands. Further, under `man modulefile`, I found that I can log to a personal file using
>> ```
>> #%Module1.0
>> module-trace on load
>> module-log trace /home/lexg/modules.log
>> ```
>> 
>> This is alright, if I execute `module load cuda`, I see some output like
>> ```
>> bash load cuda <...> = TCL_OK(0)
>> ```
>> 
>> However, subsequent module load commands simply keep appending to the first line, rather than starting on newlines. Is there any way to change this?
>> 
>> I then tried to use the `trace` command in TCL similar to the examples in https://modules.readthedocs.io/en/latest/cookbook/log-module-commands.html, but I could never get this to produce any output at all. One configuration I tried just to get *any* output upon `module load` was:
>> ```
>> proc printArguments args {
>>    foreach arg $args {
>>        puts stderr $arg
>>    }
>> }
>> 
>> trace add execution module leave printArguments
>> ```
>> 
>> I also tried outputting to files (`exec echo "BLAH" > /home/lexg/modules.log`) but to no avail.
>> 
>> I could find no real examples of `trace add execution` outside of the cookbook example linked above. Is there a way to use `trace` so I can better format the output, or am I stuck with module-log? Thanks in advance.
>> 
>> LEX GALLON
>> Research Officer, Monash e-Research Centre
>> _______________________________________________
>> Modules-interest mailing list
>> Mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/modules-interest
> 
> 
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest

Re: [Modules] Question: Modules v3.2.10 logging

[Xavier D. <xav...@gm...>]

Hi Lex,

`trace` command requires to be used in modulecmd.tcl customization
script (siteconfig.tcl). This is not available on Modules 3.

I recommend setting up an `exec` command in every modulefiles you want
to log activity.

if { [module-info mode] in {load unload} } {
    exec logger -t module "$env(USER) [module-info mode] [module-info name]"
}

The above code snippet should send a log message toward your log
system when module is loaded or unloaded.

Regards,
Xavier

Le jeu. 25 janv. 2024 à 07:26, Lex Gallon via Modules-interest
<mod...@li...> a écrit :
>
> Hi all,
>
> Firstly, apologies if this is a silly question. I've never used TCL outside of module files before.
>
> I'm on a shared system where we are stuck with Modules v3.2.10 (TCL 8.5). I've been trying to enable logging so we have a record of which modules are being loaded most often, but I've been struggling to configure it properly.
>
> Under `man module`, I found that I can add a `~/.modulerc` which gets executed on any module commands. Further, under `man modulefile`, I found that I can log to a personal file using
> ```
> #%Module1.0
> module-trace on load
> module-log trace /home/lexg/modules.log
> ```
>
> This is alright, if I execute `module load cuda`, I see some output like
> ```
> bash load cuda <...> = TCL_OK(0)
> ```
>
> However, subsequent module load commands simply keep appending to the first line, rather than starting on newlines. Is there any way to change this?
>
> I then tried to use the `trace` command in TCL similar to the examples in https://modules.readthedocs.io/en/latest/cookbook/log-module-commands.html, but I could never get this to produce any output at all. One configuration I tried just to get *any* output upon `module load` was:
> ```
> proc printArguments args {
>     foreach arg $args {
>         puts stderr $arg
>     }
> }
>
> trace add execution module leave printArguments
> ```
>
> I also tried outputting to files (`exec echo "BLAH" > /home/lexg/modules.log`) but to no avail.
>
> I could find no real examples of `trace add execution` outside of the cookbook example linked above. Is there a way to use `trace` so I can better format the output, or am I stuck with module-log? Thanks in advance.
>
> LEX GALLON
> Research Officer, Monash e-Research Centre
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest

[Modules] Question: Modules v3.2.10 logging

[Lex G. <lex...@mo...>]

Hi all,

Firstly, apologies if this is a silly question. I've never used TCL outside
of module files before.

I'm on a shared system where we are stuck with Modules v3.2.10 (TCL 8.5).
I've been trying to enable logging so we have a record of which modules are
being loaded most often, but I've been struggling to configure it properly.

Under `man module`, I found that I can add a `~/.modulerc` which gets
executed on any module commands. Further, under `man modulefile`, I found
that I can log to a personal file using
```
#%Module1.0
module-trace on load
module-log trace /home/lexg/modules.log
```

This is alright, if I execute `module load cuda`, I see some output like
```
bash load cuda <...> = TCL_OK(0)
```

However, subsequent module load commands simply keep appending to the first
line, rather than starting on newlines. *Is there any way to change this?*

I then tried to use the `trace` command in TCL similar to the examples in
https://modules.readthedocs.io/en/latest/cookbook/log-module-commands.html,
but I could never get this to produce any output at all. One configuration
I tried just to get *any* output upon `module load` was:
```
proc printArguments args {
    foreach arg $args {
        puts stderr $arg
    }
}

trace add execution module leave printArguments
```

I also tried outputting to files (`exec echo "BLAH" > /home/lexg/modules.log`)
but to no avail.

I could find no real examples of `trace add execution` outside of the
cookbook example linked above. *Is there a way to use `trace` so I can
better format the output, or am I stuck with module-log?* Thanks in advance.

*LEX GALLON*
Research Officer, Monash e-Research Centre

[Modules] Enhancing extra specifiers

[Xavier D. <xav...@gm...>]

Dear All,

"tag" extra specifier is added to search modules based on tags that
apply to them. Moreover, multiple names can now be specified on one
extra specifier criterion to select modules matching any of these
names.

Details and example are available at:

https://modules.readthedocs.io/en/latest/MIGRATING.html#enhancing-extra-specifiers

Regards,
Xavier

Re: [Modules] To catch 'module load' ERRORs

[Baudilio T. <bau...@no...>]

Hi Xavier.

That was the solution (proc) I was looking for.

Thanks a million,
Baudilio


Xavier Delaruelle <xav...@gm...> writes:

> Hello Baudilio,
>
> Error messages can be overridden with site-specific configuration script siteconfig.tcl [1].
>
> In this Tcl script file, you can supersede the definition of the procedure that report error messages (reportError), to
> change the implicit default error message.
>
> $ cat /etc/environment-modules/siteconfig.tcl
> rename __reportError __reportError_ORIG
> proc __reportError {message {severity ERROR} {sgrkey er} {incr_count 1}} {
>     if {[regexp {No default version defined for '(.*)'} $message match mod]} {
>         append message "\nHINT: Type 'module avail $mod' to get available versions"
>     }
>     __reportError_ORIG $message $severity $sgrkey $incr_count
> }
> $ module config implicit_default 0
> $ module load foo
> ERROR: No default version defined for 'foo'
>   HINT: Type 'module avail foo' to get available versions
>
> Regards,
> Xavier
>
> [1] https://modules.readthedocs.io/en/latest/module.html#site-specific-configuration
>
> Le sam. 13 janv. 2024 à 17:38, Paul Markfort <pau...@gm...> a écrit :
>
>  Why would a user do this (except for testing)?
>  The error they get is obvious (once you understand what the implicit setting does).
>  Enforcing it, is overkill.
>
>  I think the second best option is to define EXPLICIT defaults for your modules.
>  The best, is to just educate your users as to what implicit default does (and that they probably don't want to turn
>  it off).
>
>  MODULES_IMPLICIT_DEFAULT
>                 Defines (if set to 1) or not (if set to 0) an implicit default version for modules without a default
>  version explicitly  defined  (see
>                 Locating Modulefiles section in the modulefile man page).
>
>                 Without  either an explicit or implicit default version defined a module must be fully qualified
>  (version should be specified in addi-
>                 tion to its name) to get:
>
>                 • targeted by module load, switch, display, help, test and path sub-commands.
>
>                 • restored from a collection, unless already loaded in collection-specified order.
>
>                 • automatically loaded by automated module handling mechanisms (see MODULES_AUTO_HANDLING section)
>  when declared  as  module  require-
>                   ment, with prereq or module load modulefile commands.
>
>                 An error is returned in the above situations if either no explicit or implicit default version is
>  defined.
>
>  On 1/13/2024 9:50 AM, Baudilio Tejerina - NOAA Affiliate via Modules-interest wrote:
>  > Hello,
>  > 
>  > We need to enforce ‘implicit_default’ at out site.
>  > 
>  > Is it possible to catch errors such as
>  > 
>  >   $ module config implicit_default 0
>  >   $ module load foo
>  >     ERROR: No default version defined for 'foo'
>  > 
>  > 
>  > so that that I can inform users with possible, alternative, available actions?
>  > 
>  > 
>  > Thank You!
>  > Baudilio
>
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest
>

Re: [Modules] To catch 'module load' ERRORs

[Xavier D. <xav...@gm...>]

Hello Baudilio,

Error messages can be overridden with site-specific configuration script
siteconfig.tcl [1].

In this Tcl script file, you can supersede the definition of the procedure
that report error messages (reportError), to change the implicit default
error message.

$ cat /etc/environment-modules/siteconfig.tcl
rename __reportError __reportError_ORIG
proc __reportError {message {severity ERROR} {sgrkey er} {incr_count 1}} {
    if {[regexp {No default version defined for '(.*)'} $message match
mod]} {
        append message "\nHINT: Type 'module avail $mod' to get available
versions"
    }
    __reportError_ORIG $message $severity $sgrkey $incr_count
}
$ module config implicit_default 0
$ module load foo
ERROR: No default version defined for 'foo'
  HINT: Type 'module avail foo' to get available versions

Regards,
Xavier

[1]
https://modules.readthedocs.io/en/latest/module.html#site-specific-configuration

Le sam. 13 janv. 2024 à 17:38, Paul Markfort <pau...@gm...> a écrit :

>
> Why would a user do this (except for testing)?
> The error they get is obvious (once you understand what the implicit
> setting does).
> Enforcing it, is overkill.
>
>
> I think the second best option is to define EXPLICIT defaults for your
> modules.
> The best, is to just educate your users as to what implicit default does
> (and that they probably don't want to turn it off).
>
>
>
> MODULES_IMPLICIT_DEFAULT
>                Defines (if set to 1) or not (if set to 0) an implicit
> default version for modules without a default version explicitly  defined
> (see
>                Locating Modulefiles section in the modulefile man page).
>
>                Without  either an explicit or implicit default version
> defined a module must be fully qualified (version should be specified in
> addi-
>                tion to its name) to get:
>
>                • targeted by module load, switch, display, help, test and
> path sub-commands.
>
>                • restored from a collection, unless already loaded in
> collection-specified order.
>
>                • automatically loaded by automated module handling
> mechanisms (see MODULES_AUTO_HANDLING section) when declared  as  module
> require-
>                  ment, with prereq or module load modulefile commands.
>
>                An error is returned in the above situations if either no
> explicit or implicit default version is defined.
>
>
> On 1/13/2024 9:50 AM, Baudilio Tejerina - NOAA Affiliate via
> Modules-interest wrote:
> > Hello,
> >
> > We need to enforce ‘implicit_default’ at out site.
> >
> > Is it possible to catch errors such as
> >
> >   $ module config implicit_default 0
> >   $ module load foo
> >     ERROR: No default version defined for 'foo'
> >
> >
> > so that that I can inform users with possible, alternative, available
> actions?
> >
> >
> > Thank You!
> > Baudilio
>
>

Re: [Modules] To catch 'module load' ERRORs

[Paul M. <pau...@gm...>]

Why would a user do this (except for testing)?
The error they get is obvious (once you understand what the implicit setting does).
Enforcing it, is overkill.


I think the second best option is to define EXPLICIT defaults for your modules.
The best, is to just educate your users as to what implicit default does (and that they probably don't want to turn it off).



MODULES_IMPLICIT_DEFAULT
               Defines (if set to 1) or not (if set to 0) an implicit default version for modules without a default version explicitly  defined  (see
               Locating Modulefiles section in the modulefile man page).

               Without  either an explicit or implicit default version defined a module must be fully qualified (version should be specified in addi-
               tion to its name) to get:

               • targeted by module load, switch, display, help, test and path sub-commands.

               • restored from a collection, unless already loaded in collection-specified order.

               • automatically loaded by automated module handling mechanisms (see MODULES_AUTO_HANDLING section) when declared  as  module  require-
                 ment, with prereq or module load modulefile commands.

               An error is returned in the above situations if either no explicit or implicit default version is defined.


On 1/13/2024 9:50 AM, Baudilio Tejerina - NOAA Affiliate via Modules-interest wrote:
> Hello,
> 
> We need to enforce ‘implicit_default’ at out site.
> 
> Is it possible to catch errors such as
> 
>   $ module config implicit_default 0
>   $ module load foo
>     ERROR: No default version defined for 'foo'
> 
> 
> so that that I can inform users with possible, alternative, available actions?
> 
> 
> Thank You!
> Baudilio
> 
> 
> 
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest
> 

-- 
--------------------------------------------------------
The views and opinions expressed above are strictly
those of the author(s).  The content of this message has
not been reviewed nor approved by any entity whatsoever.
--------------------------------------------------------
Paul FM         Info: http://paulfm.com/~paulfm/
--------------------------------------------------------

[Modules] To catch 'module load' ERRORs

[Baudilio T. - N. A. <bau...@no...>]

Hello,

We need to enforce ‘implicit_default’ at out site.

Is it possible to catch errors such as

 $ module config implicit_default 0
 $ module load foo
   ERROR: No default version defined for 'foo'


so that that I can inform users with possible, alternative, available actions?


Thank You!
Baudilio

Re: [Modules] set-alias when module load in interactive, but 'unalias: foo: not found' when unload in non-interactive

[Xavier D. <xav...@gm...>]

Hi Myles,

Many thanks for this issue report.

I think I should adapt the shell code generated by "unset-alias", to only
execute "unalias" shell command if alias is found defined.

There is a way currently to avoid this error: let module propagate shell
aliases to sub-shells. This can be achieved for interactive script
execution by enabling "set_shell_startup" configuration option prior module
command initialization [1].

set_shell_startup defines the ENV and BASH_ENV variables and make them
point to the Modules shell initialization script. With these env variables
set, Modules shell init script is executed at start of the shell script you
run. Loaded modules are refreshed which recreates shell alias in this
sub-shell.

$ cat test
#!/bin/bash
alias foo
module purge
$ module load foo
$ alias foo
alias foo='value'
$ bash test
test: line 2: alias: foo: not found
environment: line 4: unalias: foo: not found

$ module purge
$ module config set_shell_startup 1
$ eval $(./modulecmd.tcl bash autoinit)
$ module load foo
$ bash test
alias foo='value'

Regards,
Xavier

[1]
https://modules.readthedocs.io/en/latest/module.html#envvar-MODULES_SET_SHELL_STARTUP




Le mer. 13 déc. 2023 à 07:01, Myles <smp...@gm...> a écrit :

> I have a module with a set-alias that I load for my interactive shell.
> Let's say the alias is 'foo'.
>
> Then, when I run this script from that interactive environment:
> #!/bin/bash
> module purge
>
> I am getting:
> environment: line 26: unalias: foo: not found
>
> This is because the alias is not propagated to the child process.
> Shouldn't the unalias that happens during an unload check that the context
> shell is interactive?
>
> I am using EnvModules v5.3.1.
>
> I found this discussion, but I couldn't decide what to take away from it.
>
> https://sourceforge.net/p/modules/mailman/modules-interest
> /thread/546...@at.../
>
> Thanks!
> --Myles
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest
>

[Modules] set-alias when module load in interactive, but 'unalias: foo: not found' when unload in non-interactive

[Myles <smp...@gm...>]

I have a module with a set-alias that I load for my interactive shell.
Let's say the alias is 'foo'.

Then, when I run this script from that interactive environment:
#!/bin/bash
module purge

I am getting:
environment: line 26: unalias: foo: not found

This is because the alias is not propagated to the child process. Shouldn't
the unalias that happens during an unload check that the context shell is
interactive?

I am using EnvModules v5.3.1.

I found this discussion, but I couldn't decide what to take away from it.

https://sourceforge.net/p/modules/mailman/modules-interest
/thread/546...@at.../

Thanks!
--Myles