modules-interest — Discussion forum for modules usage.

[Modules] Siteconfig hook variables

[Xavier D. <xav...@gm...>]

Hello,

Last feature of version 5.2 is finished: 4 variables are introduced
for site-specific configuration file to define specific commands and
variables in the evaluation context of modulefiles and modulercs

https://modules.readthedocs.io/en/latest/MIGRATING.html#siteconfig-hook-variables

Other kind of site-specific hooks are possible thanks to the Tcl
language. Documentation has been improved on this topic:

https://modules.readthedocs.io/en/latest/module.html#site-specific-configuration-1

Regards,
Xavier

[Modules] Improved documentation for module collections

[Xavier D. <xav...@gm...>]

Hello All,

Following a request made here to have more insight of what can be done
with module collections, the documentation of this feature has been
improved with examples:

https://modules.readthedocs.io/en/latest/module.html#collections-1

With the introduction of "Initial environment state" and "Stash
collections", it was also the good time to extend this documentation.

Regards,
Xavier

[Modules] Stashing environment

[Xavier D. <xav...@gm...>]

Hello All,

I have a new feature to unveil for the next version (5.2). It is the
ability to stash your current environment in a similar fashion than
you do with changes in a git repository.

A bunch of new module sub-commands are introduced for this feature:
stash, stashpop, stashshow, stashrm and stashclear.

Stashed environment are managed with the collection mechanism.

More details and example
at:https://modules.readthedocs.io/en/latest/MIGRATING.html#stashing-environment

Regards,
Xavier

Re: [Modules] trouble with installation configure files

[Xavier D. <xav...@gm...>]

Hi Corey,

This is not an issue I am aware of. I have just tested to run the
./configure && make sequence on the latest released version (5.1.1)
from a tcsh shell on a Fedora Linux system. It works well for me.

Could you send me the output obtained when running ./configure and any
other log or traces you get.

Note that you can also install and use Modules without the Tcl
Envmodules lib with the --disable-libtclenvmodules configure option.

Regards,
Xavier


Le mar. 1 nov. 2022 à 21:01, Corey Coughlin
<cor...@ac...> a écrit :
>
> Hi all,
>
> Not sure where to get help with this, but it looks like the mailing list is it, so here I am.  Basically, I’m doing the configure step on linux (CentOs 6.10) and I’m having a lot of trouble when it drops into the lib/configure file.  Specifically, it either dies (because my default shell is tcsh) or it gets stuck in an infinite loop (if I set CONFIG_SHELL to bash or sh).  I’ve turned the sh –xv options to take a closer look, but I’m not super familiar with Bourne shell scripting, so it’s easy for me to get lost.  There’s a comment around line 512 that seems to imply that there’s a way to keep it out of the infinite loop, but it doesn’t seem to be working for me.  Is this a known issue, or do I need to set another environment variable to get this to work?  I can show the traces from the –xv option if that helps, wanted to keep this clear to start with.  Thanks for any help!
>
>
>
>                 -------- Corey
>
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest

[Modules] trouble with installation configure files

[Corey C. <cor...@ac...>]

Hi all,
Not sure where to get help with this, but it looks like the mailing list is it, so here I am.  Basically, I'm doing the configure step on linux (CentOs 6.10) and I'm having a lot of trouble when it drops into the lib/configure file.  Specifically, it either dies (because my default shell is tcsh) or it gets stuck in an infinite loop (if I set CONFIG_SHELL to bash or sh).  I've turned the sh -xv options to take a closer look, but I'm not super familiar with Bourne shell scripting, so it's easy for me to get lost.  There's a comment around line 512 that seems to imply that there's a way to keep it out of the infinite loop, but it doesn't seem to be working for me.  Is this a known issue, or do I need to set another environment variable to get this to work?  I can show the traces from the -xv option if that helps, wanted to keep this clear to start with.  Thanks for any help!

                -------- Corey

[Modules] Code of Conduct for Modules

[Xavier D. <xav...@gm...>]

Hello All,

Following good practices, a Code of Conduct has been set for the
project and all its community spaces (web site, github/sourceforge
project, mailing-list, twitter account, documentation portal, etc):

https://github.com/cea-hpc/modules/blob/main/CODE_OF_CONDUCT.md

It is based on the widely adopted Contributor Covenant.

Regards,
Xavier

[Modules] Default branch of git repository renamed

[Xavier D. <xav...@gm...>]

Hello All,

To adopt more inclusive terminology [1], the default branch of the git
repository has been renamed from master to main.

For those of you who have a local copy of the git repository of
Modules, apply the following commands to update your local clone:

$ git branch -m master main
$ git branch -m c-master c-main
$ git fetch origin
$ git branch -u origin/main main
$ git branch -u origin/c-main c-main
$ git remote set-head origin -a
$ git remote prune origin

Regards,
Xavier

[1] https://github.com/github/renaming

[Modules] Initial environment

[Xavier D. <xav...@gm...>]

Hello All,

`reset` sub-command is introduced for next version (5.2): like done on
Lmod, it restores initial user environment. Here it relies on the
collection mechanism with initial environment captured in an
environment variable.

Details and examples available at
https://modules.readthedocs.io/en/latest/MIGRATING.html#initial-environment

Regards,
Xavier

Re: [Modules] make test debugging questions/comments

[Xavier D. <xav...@gm...>]

Le mar. 19 juil. 2022 à 05:36, Xavier Delaruelle <
xav...@gm...> a écrit :

> Hello Ben,
>
> Le lun. 18 juil. 2022 à 15:11, Ben Bowers <nan...@gm...> a écrit :
>
>> Hello modules developers/maintainers,
>>
>> Some questions/comments about the tests included with modules:
>>
>> 1) it takes me about 12 minutes of runtime to run the tests.  Is there
>> some option to parallelize or get through the tests quicker?  (I do realize
>> there are ~17000 tests/asserts)
>>
>
> There is a lightweight version of the testsuite (~1000 tests, takes
> ~1minute to run) that could be invoked with:
>
> $ make test QUICKTEST=1
>
> Then there is the "mt" helper script that enables to run just some parts
> of the testsuite
>
>
>> 2) Per comment  #1 above, there appears to be a helper script named "mt"
>> - I seemed to remember this from a long time ago... This doesn't look like
>> it is documented in the CONTRIBUTING.rst document, but seems like it would
>> be a worthy addition, because running the full test suite takes a long
>> time.  "mt" has neither a help message nor any in-script documentation
>> examples on how to run and it took more than a few tries to figure out how
>> to get it to run properly.  Any thoughts on where MT should be documented -
>> in the script vs. in the contributing document, or both?
>>
>
> In the contributing document and it should also get a -h/--help option to
> output a usage message.
>
>
>> 3) When working on a branch, I got unexpected failures from ~130 tests
>> because my local branch name (test_mode_mimic) used underscores, which
>> caused tests to fail since the regex expects the version to only contain
>> alphanumeric characters, or dot / plus / dash :
>> ERR[65]: 'Modules Release 5.1.1+test_mode_mimic-107-g5b9dbfac (2022-07-17)
>> EXP[53]: '^Modules Release [0-9a-zA-Z\.\+\-]+ \([0-9\-]{10}\)
>> I was able to correct this by renaming my branch to "test-mode-mimic".  I
>> did not notice this restriction documented anywhere, would it be
>> appropriate to add to the CONTRIBUTING.rst ?
>> Or should the regex be updated to allow underscores in the version name?
>>
>
> The regexp should be updated to allow underscores in the version name.
>
>
>> 4) Slightly off topic - It would be nice if the makefile had a help
>> target.  There are ways to embed special comments in makefiles to make it
>> self-documenting, per Self-Documented Makefile (marmelab.com)
>> <https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html>
>>
>
> I would be interesting to add such help target. It would be great if the
> relative code could only rely on 'sed', as the Makefile does not depend on
> grep and awk currently.
>
>
>> I could take a crack at these if they are of interest.
>>
>
> It would be great for sure. Do you mean 2, 3 and 4 or just 4 ?
>
> Best regards,
> Xavier
>

Some update on this topic. I have implemented 2 and 3. So 'script/mt' is
now documented on CONTRIBUTING guide:


https://modules.readthedocs.io/en/latest/CONTRIBUTING.html#running-the-tests-1



Also 'mt' (and other utilities in script directory) now have a usage
message that can be triggered with --help/-h option.

Regards,

Xavier

[Modules] mod-to-sh sub-command

[Xavier D. <xav...@gm...>]

Hello All,

A new sub-command is added for next version (5.2): mod-to-sh. It evaluates
modulefiles passed as argument and produces code for specified shell.

$ cat /path/to/modulefiles/foo
#%Module
setenv FOO value
set-function foo {echo foo}$ module mod-to-sh bash foo
FOO=value; export FOO;
foo () { echo foo; }; export -f foo;

More details at
https://modules.readthedocs.io/en/latest/MIGRATING.html#mod-to-sh-sub-command

Regards,
Xavier

Re: [Modules] Make set-alias create function instead of alias

[Gretton, L. <lia...@le...>]

I really had tried quotes in place of the braces, but must have not reloaded the module properly.

Thanks again, this is all working nicely now.

Re: [Modules] Make set-alias create function instead of alias

[Xavier D. <xav...@gm...>]

This due to the curly braces used around the function content. In Tcl curly
braces mean the content is set verbatim, no variable resolution occurs.

Use double quotes instead curly braces and it should be ok:

$ cat test
#%Module
set ppath     /path/to/apps
set package   app
set version   x.y.z
set-function appalias "$ppath/$package/$version/scripts/thing --option"
$ module load ./test
$ declare -f appalias
appalias ()
{
    /path/to/apps/app/x.y.z/scripts/thing --option
}

Regards,
Xavier

Le mer. 10 août 2022 à 18:40, Gretton, Liam <lia...@le...>
a écrit :

> Thanks for the explanation.
>
>
>
> I'm almost there with this, just having problems passing Tcl variables
> into the set-function definition.
>
>
>
> Our module files define the application path at the start like so:
>
>
>
> set ppath     /path/to/apps
>
> set package   app
>
> set version   x.y.z
>
>
>
> In set-alias I can just refer to those as so:
>
>
>
> set-alias appalias "$ppath/$package/$version/scripts/thing --option"
>
>
>
> But in set-function everything within the braces is passed through to the
> shell literally and the variable values aren't passed:
>
>
>
> set-function appalias  { $ppath/$package/$version/scripts/thing --option }
>
>
>
> declare -f appalias
>
> appalias ()
>
> {
>
>   $ppath/$package/$version/scripts/thing --option
>
> }
>
>
>
> …instead of
>
>
>
> declare -f appalias
>
> appalias ()
>
> {
>
>   /path/to/apps/app/x.y.z/scripts/thing --option
>
> }
>
>
>
> How can I get those variables declared within the module to be
> dereferenced in the function definition? This could be just my Tcl-foo
> being inadequate.
>
>
>
> Thanks,
>
> Liam
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest
>

Re: [Modules] Make set-alias create function instead of alias

[Gretton, L. <lia...@le...>]

Thanks for the explanation.

I'm almost there with this, just having problems passing Tcl variables into the set-function definition.

Our module files define the application path at the start like so:

set ppath     /path/to/apps
set package   app
set version   x.y.z

In set-alias I can just refer to those as so:

set-alias appalias "$ppath/$package/$version/scripts/thing --option"

But in set-function everything within the braces is passed through to the shell literally and the variable values aren't passed:

set-function appalias  { $ppath/$package/$version/scripts/thing --option }

declare -f appalias
appalias ()
{
  $ppath/$package/$version/scripts/thing --option
}

...instead of

declare -f appalias
appalias ()
{
  /path/to/apps/app/x.y.z/scripts/thing --option
}

How can I get those variables declared within the module to be dereferenced in the function definition? This could be just my Tcl-foo being inadequate.

Thanks,

Liam

[Modules] Shell support

[Xavier D. <xav...@gm...>]

Hello All,

I have just added a "Shell support" section in modulefile man page to
describe what shells are supported by Modules and more specifically what
kind of environment changes are supported by these shells (variable,
function changes, ...):

https://modules.readthedocs.io/en/latest/modulefile.html#shell-support

Regards,
Xavier

Re: [Modules] Make set-alias create function instead of alias

[Paul M. <pau...@gm...>]

I just did a test (of aliases in bash).

I created a script with these contents:

#! /bin/sh
alias show_ok='echo OK'
show_ok

When run, it works,
If I change it to
	#! /bin/bash
, it doesn't work.

This of course works:
#! /bin/bash --norc
shopt -s expand_aliases
alias show_ok='echo OK'
show_ok

My advice:
In your module, check if the shell is bash, if it is, then add this:
	puts "shopt -s expand_aliases"
(this will send that command to the bash shell to evaluate, hopefully it is interpreted before any aliases are used).
You will then be able to use aliases.

An alternative is to start your bash scripts that use aliases with:
shopt -s expand_aliases
(or make them all /bin/sh scripts - on Linux, this works in most cases - on other OSes with a real bourne shell, this is fraught with problems).

Is it possible there was a change in the way bash works that changed the way your modules work (possibly a security fix).  Or perhaps you changed from /bin/sh to /bin/bash (even though both are really the same shell on Linux).




  


On 2022-07-29 03:35 PM, Xavier Delaruelle wrote:
> Hello Liam,
> 
> This set-alias change occurred on version 4.0 [1]. The goal was to make
> set-alias consistent: making an alias and only an alias.
> 
> I see that I need to update the set-alias documentation in the man-page
> [2], as it still refer to the behavior of version 3.2.
> 
> Since version 4.2 set-function enables to define shell functions.
> 
> As you say the function definition has to be written in target shell's
> syntax, but it is also true for alias definition. Hopefully an alias can be
> implemented as a shell function with very simple shell code that can work
> over different kind of shells.
> 
> However if code has to be specific for every supported shells, some Tcl
> code like the following could help to provide a specific definition for
> each supported shell within a single modulefile:
> 
> switch -- [module-info shelltype] {
>     sh {
>        set-function foo {sh code here}
>     }
>     csh {
>        set-alias foo {csh code here}
>     }
>     fish {
>        set-function foo {fish code here}
>     }
> }
> 
> Regards,
> Xavier
> 
> [1]
> https://modules.readthedocs.io/en/stable/changes.html#modules-specific-tcl-commands
> [2] https://modules.readthedocs.io/en/stable/modulefile.html#mfcmd-set-alias
> 
> Le ven. 29 juil. 2022 à 21:03, Gretton, Liam <lia...@le...>
> a écrit :
> 
>> We're using Modules 4.2.1 on a HPC system.
>>
>>
>>
>> Using set-alias in a module always creates an alias in the target shell
>> (bash mostly in our case), but that doesn't work for non-interactive
>> shells; the alias is defined but without (for bash) shopt -s
>> expand_aliases being executed, they're not usable.
>>
>>
>>
>> The manpage states that set-alias will set an alias or function. Is there
>> a way to force it to create a function instead of an alias?
>>
>>
>>
>> set-function isn't much use as the definition has to be written in the
>> target shell's syntax, and one of the benefits of using module files is
>> that they're shell-agnostic.
>>
>>
>>
>> I'm fairly certain this is new behaviour for 4.2.1 or thereabouts. Until
>> about a year ago we were running an older version of modulefiles and I'm pretty
>> certain we didn't get this problem.
>>
>>
>>
>> Thanks,
>>
>>
>> Liam
>>
> 
> 
> 
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest
> 

-- 
--------------------------------------------------------
The views and opinions expressed above are strictly
those of the author(s).  The content of this message has
not been reviewed nor approved by any entity whatsoever.
--------------------------------------------------------
Paul FM         Info: http://paulfm.com/~paulfm/
--------------------------------------------------------

Re: [Modules] Make set-alias create function instead of alias

[Xavier D. <xav...@gm...>]

Hello Liam,

This set-alias change occurred on version 4.0 [1]. The goal was to make
set-alias consistent: making an alias and only an alias.

I see that I need to update the set-alias documentation in the man-page
[2], as it still refer to the behavior of version 3.2.

Since version 4.2 set-function enables to define shell functions.

As you say the function definition has to be written in target shell's
syntax, but it is also true for alias definition. Hopefully an alias can be
implemented as a shell function with very simple shell code that can work
over different kind of shells.

However if code has to be specific for every supported shells, some Tcl
code like the following could help to provide a specific definition for
each supported shell within a single modulefile:

switch -- [module-info shelltype] {
   sh {
      set-function foo {sh code here}
   }
   csh {
      set-alias foo {csh code here}
   }
   fish {
      set-function foo {fish code here}
   }
}

Regards,
Xavier

[1]
https://modules.readthedocs.io/en/stable/changes.html#modules-specific-tcl-commands
[2] https://modules.readthedocs.io/en/stable/modulefile.html#mfcmd-set-alias

Le ven. 29 juil. 2022 à 21:03, Gretton, Liam <lia...@le...>
a écrit :

> We're using Modules 4.2.1 on a HPC system.
>
>
>
> Using set-alias in a module always creates an alias in the target shell
> (bash mostly in our case), but that doesn't work for non-interactive
> shells; the alias is defined but without (for bash) shopt -s
> expand_aliases being executed, they're not usable.
>
>
>
> The manpage states that set-alias will set an alias or function. Is there
> a way to force it to create a function instead of an alias?
>
>
>
> set-function isn't much use as the definition has to be written in the
> target shell's syntax, and one of the benefits of using module files is
> that they're shell-agnostic.
>
>
>
> I'm fairly certain this is new behaviour for 4.2.1 or thereabouts. Until
> about a year ago we were running an older version of modulefiles and I'm pretty
> certain we didn't get this problem.
>
>
>
> Thanks,
>
>
> Liam
>

[Modules] Make set-alias create function instead of alias

[Gretton, L. <lia...@le...>]

We're using Modules 4.2.1 on a HPC system.

Using set-alias in a module always creates an alias in the target shell (bash mostly in our case), but that doesn't work for non-interactive shells; the alias is defined but without (for bash) shopt -s expand_aliases being executed, they're not usable.

The manpage states that set-alias will set an alias or function. Is there a way to force it to create a function instead of an alias?

set-function isn't much use as the definition has to be written in the target shell's syntax, and one of the benefits of using module files is that they're shell-agnostic.

I'm fairly certain this is new behaviour for 4.2.1 or thereabouts. Until about a year ago we were running an older version of modulefiles and I'm pretty certain we didn't get this problem.

Thanks,

Liam

Liam Gretton
Systems Specialist

Estates and Digital Services,
University of Leicester, University Road, Leicester, LE1 7RH, UK

e:  lia...@le...<mailto:lia...@le...>
w: www.le.ac.uk<http://www.le.ac.uk/>

[cid:image001.png@01D8A365.C78CAFA0]

Follow us on Twitter<https://twitter.com/uniofleicester> or visit our Facebook<http://www.facebook.com/uniofleicester> page

[Modules] Linting modulefiles

[Xavier D. <xav...@gm...>]

Hello All,

Another feature for next version (5.2): ability to check the syntax of your
modulefiles with a new sub-command: module lint.

Coupled to Nagelfar, the static analyzer for Tcl (
http://nagelfar.sourceforge.net/), you can now check the Tcl syntax of your
modules and also the specific syntax of modulefile commands.

More details and examples at:

https://modules.readthedocs.io/en/latest/MIGRATING.html#linting-modulefiles

Work has also been done to make Nagelfar available as a package for Fedora
and EL environments:
  https://src.fedoraproject.org/rpms/nagelfar

Regards,
Xavier

[Modules] --timer command-line switch

[Xavier D. <xav...@gm...>]

Hello All,

The --timer command-line switch is introduced in next version (5.2). It
reports the total execution time of the module command. It also times every
internal procedure call when used in debug mode.

See https://modules.readthedocs.io/en/latest/module.html#cmdoption-timer

Regards,
Xavier

Re: [Modules] make test debugging questions/comments

[Xavier D. <xav...@gm...>]

Hello Ben,

Le lun. 18 juil. 2022 à 15:11, Ben Bowers <nan...@gm...> a écrit :

> Hello modules developers/maintainers,
>
> Some questions/comments about the tests included with modules:
>
> 1) it takes me about 12 minutes of runtime to run the tests.  Is there
> some option to parallelize or get through the tests quicker?  (I do realize
> there are ~17000 tests/asserts)
>

There is a lightweight version of the testsuite (~1000 tests, takes
~1minute to run) that could be invoked with:

$ make test QUICKTEST=1

Then there is the "mt" helper script that enables to run just some parts of
the testsuite


> 2) Per comment  #1 above, there appears to be a helper script named "mt" -
> I seemed to remember this from a long time ago... This doesn't look like it
> is documented in the CONTRIBUTING.rst document, but seems like it would be
> a worthy addition, because running the full test suite takes a long time.
> "mt" has neither a help message nor any in-script documentation examples on
> how to run and it took more than a few tries to figure out how to get it to
> run properly.  Any thoughts on where MT should be documented - in the
> script vs. in the contributing document, or both?
>

In the contributing document and it should also get a -h/--help option to
output a usage message.


> 3) When working on a branch, I got unexpected failures from ~130 tests
> because my local branch name (test_mode_mimic) used underscores, which
> caused tests to fail since the regex expects the version to only contain
> alphanumeric characters, or dot / plus / dash :
> ERR[65]: 'Modules Release 5.1.1+test_mode_mimic-107-g5b9dbfac (2022-07-17)
> EXP[53]: '^Modules Release [0-9a-zA-Z\.\+\-]+ \([0-9\-]{10}\)
> I was able to correct this by renaming my branch to "test-mode-mimic".  I
> did not notice this restriction documented anywhere, would it be
> appropriate to add to the CONTRIBUTING.rst ?
> Or should the regex be updated to allow underscores in the version name?
>

The regexp should be updated to allow underscores in the version name.


> 4) Slightly off topic - It would be nice if the makefile had a help
> target.  There are ways to embed special comments in makefiles to make it
> self-documenting, per Self-Documented Makefile (marmelab.com)
> <https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html>
>

I would be interesting to add such help target. It would be great if the
relative code could only rely on 'sed', as the Makefile does not depend on
grep and awk currently.


> I could take a crack at these if they are of interest.
>

It would be great for sure. Do you mean 2, 3 and 4 or just 4 ?

Best regards,
Xavier

[Modules] make test debugging questions/comments

[Ben B. <nan...@gm...>]

Hello modules developers/maintainers,

Some questions/comments about the tests included with modules:

1) it takes me about 12 minutes of runtime to run the tests.  Is there some
option to parallelize or get through the tests quicker?  (I do realize
there are ~17000 tests/asserts)

2) Per comment  #1 above, there appears to be a helper script named "mt" -
I seemed to remember this from a long time ago... This doesn't look like it
is documented in the CONTRIBUTING.rst document, but seems like it would be
a worthy addition, because running the full test suite takes a long time.
"mt" has neither a help message nor any in-script documentation examples on
how to run and it took more than a few tries to figure out how to get it to
run properly.  Any thoughts on where MT should be documented - in the
script vs. in the contributing document, or both?

3) When working on a branch, I got unexpected failures from ~130 tests
because my local branch name (test_mode_mimic) used underscores, which
caused tests to fail since the regex expects the version to only contain
alphanumeric characters, or dot / plus / dash :
ERR[65]: 'Modules Release 5.1.1+test_mode_mimic-107-g5b9dbfac (2022-07-17)
EXP[53]: '^Modules Release [0-9a-zA-Z\.\+\-]+ \([0-9\-]{10}\)
I was able to correct this by renaming my branch to "test-mode-mimic".  I
did not notice this restriction documented anywhere, would it be
appropriate to add to the CONTRIBUTING.rst ?
Or should the regex be updated to allow underscores in the version name?

4) Slightly off topic - It would be nice if the makefile had a help
target.  There are ways to embed special comments in makefiles to make it
self-documenting, per Self-Documented Makefile (marmelab.com)
<https://marmelab.com/blog/2016/02/29/auto-documented-makefile.html>

I could take a crack at these if they are of interest.

Re: [Modules] Best way to extract changes to environment from module file.

[Erik P. <eri...@si...>]

Thanks Jesse for those suggestions! I’m not too great at TCL so your input is very helpful. I think that tcl_platform will be a good replacement for the current solution.

> On Jul 14, 2022, at 3:33 PM, Jesse Lucas via Modules-interest <mod...@li...> wrote:
> 
> Wrap the source statement in a test for the os-detect.tc <http://os-detect.tc/> file's existence with [file exists ...].
> 
> Depending on the depth of information you need about the OS, maybe you can get rid of that file and just use tcl_platform global array.  https://wiki.tcl-lang.org/page/tcl_platform <https://wiki.tcl-lang.org/page/tcl_platform>
> 
> -J
> 
> On Thu, Jul 14, 2022 at 5:14 PM Erik Parawell <eri...@si... <mailto:eri...@si...>> wrote:
> Thanks for the input!
> 
> I will just import the module file into the container and use that to setup the environment. As was said, I would just end up reimplementing a lot of what modulecmd already does.
> That `module show` command is perfect for parsing my MODULE_BASE_DIR env var that is set in the file.
> 
> I do have one problematic line that is in most of my files though.
> ```
> # obtain operating system detection functions
> source [file dirname $ModulesCurrentModulefile]/../../os-detect.tc <http://os-detect.tc/>
> ```
> And the `/../../os-detect.tc <http://os-detect.tc/>` part can be up or down n number of directories since it is it just a relative path and some modules are one, two or three deep. I don’t suppose there is some way to “disable” that line without programmatically removing it from the file no? 
> 
> 
>> On Jul 14, 2022, at 11:45 AM, Sternberg, Michael G. via Modules-interest <mod...@li... <mailto:mod...@li...>> wrote:
>> 
>> The Environment Modules approach and its command language are designed to solve this exact problem. The actual modulefile contents are a very concise way to (a) describe, (b) apply, and (c) revert such changes to environment variables [+ a bit more]. You can see the commands by inspection of the modulefile, or by running "module show modulename".
>> 
>> – If by "shell loadable .env file" you mean solely and explicitly *setting* or *unsetting* variables to static values, you preclude prepend/append operations that are needed for most PATH-type variables. If you allow for variable *modification*, you essentially would have to re-implement what Modules does.
>> 
>> – If you mean "source-able shell code", then why not have that code be "module load modulename", ensuring that modulefiles are properly provisioned and Environment Modules is initialized for the relevant shell's profile and rc-files?
>> 
>> 
>> With best regards,
>> -- 
>> Michael Sternberg, Ph.D.
>> Principal Scientific Computing Administrator
>> Center for Nanoscale Materials
>> Argonne National Laboratory
>> 
>> 
>>> On Jul 14, 2022, at 13:06, Erik Parawell <eri...@si... <mailto:eri...@si...>> wrote:
>>> I have a use case where I need to figure out what changed in the environment from performing a `module load` and then export those changes to a shell loadable .env file. I have toyed with a couple ideas as listed below. But hopefully you guys might know of a easier path for me to take.
>>> 
>>> 1) Creating a program to load the module and then extract the changes of the environment and save that to a file.
>>> 2) Importing the module file into the container and letting it do the rest. The caveat is that the module still needs to be loaded beforehand to read the MODULE_BASE_DIR env var we have in each module to determine where the squashfs file needs to be created from and where the mount point would need to be.
>>> 3) Hopefully there is something easier or more maintainable than what I have been thinking of.
>>> <Squashfs tools(2).png>
>> _______________________________________________
>> Modules-interest mailing list
>> Mod...@li... <mailto:Mod...@li...>
>> https://lists.sourceforge.net/lists/listinfo/modules-interest <https://lists.sourceforge.net/lists/listinfo/modules-interest>
> 
> _______________________________________________
> Modules-interest mailing list
> Mod...@li... <mailto:Mod...@li...>
> https://lists.sourceforge.net/lists/listinfo/modules-interest <https://lists.sourceforge.net/lists/listinfo/modules-interest>
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest

Re: [Modules] Best way to extract changes to environment from module file.

[Jesse L. <je...@bl...>]

Wrap the source statement in a test for the os-detect.tc file's existence
with [file exists ...].

Depending on the depth of information you need about the OS, maybe you can
get rid of that file and just use tcl_platform global array.
https://wiki.tcl-lang.org/page/tcl_platform

-J

On Thu, Jul 14, 2022 at 5:14 PM Erik Parawell <eri...@si...>
wrote:

> Thanks for the input!
>
> I will just import the module file into the container and use that to
> setup the environment. As was said, I would just end up reimplementing a
> lot of what modulecmd already does.
> That `module show` command is perfect for parsing my MODULE_BASE_DIR env
> var that is set in the file.
>
> I do have one problematic line that is in most of my files though.
> ```
> # obtain operating system detection functions
> source [file dirname $ModulesCurrentModulefile]/../../os-detect.tc
> ```
> And the `/../../os-detect.tc` part can be up or down n number of
> directories since it is it just a relative path and some modules are one,
> two or three deep. I don’t suppose there is some way to “disable” that line
> without programmatically removing it from the file no?
>
>
> On Jul 14, 2022, at 11:45 AM, Sternberg, Michael G. via Modules-interest <
> mod...@li...> wrote:
>
> The Environment Modules approach and its command language are designed to
> solve this exact problem. The actual modulefile contents are a very concise
> way to (a) describe, (b) apply, and (c) revert such changes to environment
> variables [+ a bit more]. You can see the commands by inspection of the
> modulefile, or by running "module show modulename".
>
> – If by "shell loadable .env file" you mean solely and explicitly
> *setting* or *unsetting* variables to static values, you preclude
> prepend/append operations that are needed for most PATH-type variables. If
> you allow for variable *modification*, you essentially would have to
> re-implement what Modules does.
>
> – If you mean "source-able shell code", then why not have that code be
> "module load modulename", ensuring that modulefiles are properly
> provisioned and Environment Modules is initialized for the relevant shell's
> profile and rc-files?
>
>
> With best regards,
> --
> Michael Sternberg, Ph.D.
> Principal Scientific Computing Administrator
> Center for Nanoscale Materials
> Argonne National Laboratory
>
>
> On Jul 14, 2022, at 13:06, Erik Parawell <eri...@si...> wrote:
> I have a use case where I need to figure out what changed in the
> environment from performing a `module load` and then export those changes
> to a shell loadable .env file. I have toyed with a couple ideas as listed
> below. But hopefully you guys might know of a easier path for me to take.
>
> 1) Creating a program to load the module and then extract the changes of
> the environment and save that to a file.
> 2) Importing the module file into the container and letting it do the
> rest. The caveat is that the module still needs to be loaded beforehand to
> read the MODULE_BASE_DIR env var we have in each module to determine where
> the squashfs file needs to be created from and where the mount point would
> need to be.
> 3) Hopefully there is something easier or more maintainable than what I
> have been thinking of.
> <Squashfs tools(2).png>
>
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest
>
>
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest
>

Re: [Modules] Best way to extract changes to environment from module file.

[Erik P. <eri...@si...>]

Thanks for the input!

I will just import the module file into the container and use that to setup the environment. As was said, I would just end up reimplementing a lot of what modulecmd already does.
That `module show` command is perfect for parsing my MODULE_BASE_DIR env var that is set in the file.

I do have one problematic line that is in most of my files though.
```
# obtain operating system detection functions
source [file dirname $ModulesCurrentModulefile]/../../os-detect.tc
```
And the `/../../os-detect.tc` part can be up or down n number of directories since it is it just a relative path and some modules are one, two or three deep. I don’t suppose there is some way to “disable” that line without programmatically removing it from the file no? 


> On Jul 14, 2022, at 11:45 AM, Sternberg, Michael G. via Modules-interest <mod...@li...> wrote:
> 
> The Environment Modules approach and its command language are designed to solve this exact problem. The actual modulefile contents are a very concise way to (a) describe, (b) apply, and (c) revert such changes to environment variables [+ a bit more]. You can see the commands by inspection of the modulefile, or by running "module show modulename".
> 
> – If by "shell loadable .env file" you mean solely and explicitly *setting* or *unsetting* variables to static values, you preclude prepend/append operations that are needed for most PATH-type variables. If you allow for variable *modification*, you essentially would have to re-implement what Modules does.
> 
> – If you mean "source-able shell code", then why not have that code be "module load modulename", ensuring that modulefiles are properly provisioned and Environment Modules is initialized for the relevant shell's profile and rc-files?
> 
> 
> With best regards,
> -- 
> Michael Sternberg, Ph.D.
> Principal Scientific Computing Administrator
> Center for Nanoscale Materials
> Argonne National Laboratory
> 
> 
>> On Jul 14, 2022, at 13:06, Erik Parawell <eri...@si...> wrote:
>> I have a use case where I need to figure out what changed in the environment from performing a `module load` and then export those changes to a shell loadable .env file. I have toyed with a couple ideas as listed below. But hopefully you guys might know of a easier path for me to take.
>> 
>> 1) Creating a program to load the module and then extract the changes of the environment and save that to a file.
>> 2) Importing the module file into the container and letting it do the rest. The caveat is that the module still needs to be loaded beforehand to read the MODULE_BASE_DIR env var we have in each module to determine where the squashfs file needs to be created from and where the mount point would need to be.
>> 3) Hopefully there is something easier or more maintainable than what I have been thinking of.
>> <Squashfs tools(2).png>
> _______________________________________________
> Modules-interest mailing list
> Mod...@li...
> https://lists.sourceforge.net/lists/listinfo/modules-interest

Re: [Modules] Best way to extract changes to environment from module file.

[Paul M. <pau...@gm...>]

Correction.
Module show, just shows you what the module file will do (in module language).

Or, as already posted by someone else - you could just read the module file.


On 2022-07-14 01:45 PM, Paul Markfort wrote:
> 
> make sure you are using the shell you will be creating the .env file for
> Then run:
> module show {modulename}
> 
> The output is suitable to directly use in the .env file.
> 
> On 2022-07-14 01:00 PM, Erik Parawell wrote:
>> Hi all,
>>
>> I have a use case where I need to figure out what changed in the environment from performing a `module load` and then export those changes to a shell loadable .env file. I have toyed with a couple ideas as listed below. But hopefully you guys might know of a easier path for me to take.
>>
>> 1) Creating a program to load the module and then extract the changes of the environment and save that to a file.
>> 2) Importing the module file into the container and letting it do the rest. The caveat is that the module still needs to be loaded beforehand to read the MODULE_BASE_DIR env var we have in each module to determine where the squashfs file needs to be created from and where the mount point would need to be.
>>
>> 3) Hopefully there is something easier or more maintainable than what I have been thinking of.
>>
>>
>>
>>
>> _______________________________________________
>> Modules-interest mailing list
>> Mod...@li...
>> https://lists.sourceforge.net/lists/listinfo/modules-interest
>>
> 

-- 
--------------------------------------------------------
The views and opinions expressed above are strictly
those of the author(s).  The content of this message has
not been reviewed nor approved by any entity whatsoever.
--------------------------------------------------------
Paul FM         Info: http://paulfm.com/~paulfm/
--------------------------------------------------------