lmod-users Mailing List for Lmod (Page 4)
A Lua based environment module system that reads TCL modulefiles.
Brought to you by:
rtmclay
Menu
▾
▴
lmod-users — A list to discuss using/deploying Lmod
You can subscribe to this list here.
| 2012 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(7) |
Nov
(37) |
Dec
(9) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2013 |
Jan
(22) |
Feb
(9) |
Mar
(28) |
Apr
(13) |
May
(30) |
Jun
(49) |
Jul
(24) |
Aug
(20) |
Sep
(21) |
Oct
(71) |
Nov
(18) |
Dec
(26) |
| 2014 |
Jan
(57) |
Feb
(73) |
Mar
(39) |
Apr
(74) |
May
(55) |
Jun
(27) |
Jul
(25) |
Aug
(59) |
Sep
(43) |
Oct
(43) |
Nov
(38) |
Dec
(8) |
| 2015 |
Jan
(32) |
Feb
(38) |
Mar
(23) |
Apr
(15) |
May
(8) |
Jun
(45) |
Jul
(43) |
Aug
(6) |
Sep
(43) |
Oct
(58) |
Nov
(12) |
Dec
(31) |
| 2016 |
Jan
(21) |
Feb
(20) |
Mar
(12) |
Apr
(15) |
May
(18) |
Jun
(28) |
Jul
(3) |
Aug
(30) |
Sep
(31) |
Oct
(23) |
Nov
(49) |
Dec
(49) |
| 2017 |
Jan
(90) |
Feb
(57) |
Mar
(46) |
Apr
(35) |
May
(43) |
Jun
(23) |
Jul
(40) |
Aug
(51) |
Sep
(22) |
Oct
(21) |
Nov
(29) |
Dec
(29) |
| 2018 |
Jan
(7) |
Feb
(22) |
Mar
(16) |
Apr
(17) |
May
(18) |
Jun
(16) |
Jul
(16) |
Aug
(8) |
Sep
(29) |
Oct
(52) |
Nov
(24) |
Dec
(29) |
| 2019 |
Jan
(11) |
Feb
(13) |
Mar
(22) |
Apr
(43) |
May
(23) |
Jun
(7) |
Jul
(14) |
Aug
(27) |
Sep
(9) |
Oct
(8) |
Nov
(36) |
Dec
(58) |
| 2020 |
Jan
(29) |
Feb
(13) |
Mar
(49) |
Apr
(16) |
May
(7) |
Jun
(27) |
Jul
(12) |
Aug
(21) |
Sep
(11) |
Oct
(10) |
Nov
(12) |
Dec
(4) |
| 2021 |
Jan
(23) |
Feb
(10) |
Mar
(8) |
Apr
(16) |
May
(15) |
Jun
(19) |
Jul
(19) |
Aug
(11) |
Sep
(28) |
Oct
(25) |
Nov
(3) |
Dec
(18) |
| 2022 |
Jan
(17) |
Feb
(41) |
Mar
(19) |
Apr
(36) |
May
(40) |
Jun
(6) |
Jul
(17) |
Aug
(16) |
Sep
(12) |
Oct
(8) |
Nov
(12) |
Dec
(4) |
| 2023 |
Jan
(6) |
Feb
(7) |
Mar
(26) |
Apr
(9) |
May
(3) |
Jun
(6) |
Jul
(15) |
Aug
(11) |
Sep
(3) |
Oct
(4) |
Nov
(6) |
Dec
(17) |
| 2024 |
Jan
(13) |
Feb
(9) |
Mar
(5) |
Apr
(6) |
May
(6) |
Jun
(6) |
Jul
(12) |
Aug
(17) |
Sep
(14) |
Oct
(15) |
Nov
(20) |
Dec
(7) |
| 2025 |
Jan
(11) |
Feb
(1) |
Mar
(6) |
Apr
(14) |
May
(17) |
Jun
(4) |
Jul
|
Aug
(4) |
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Robert M. <mc...@ta...> - 2024-11-11 19:59:27
|
Tomorrow we will have our November meeting at 9:30 US Central (15:30 UTC) There are many things to discuss at the meeting. Among them are: * Q/A * Support for TCL 9.0 (Kinda) * Support for non-reversible functions like: setenv, load, etc. * Support for controlling hidden module display with env var. LMOD_SHOW_HIDDEN * Bug fixes for how hidden modules are handled when $MODULEPATH changes. * Show and Tell at SC'24 You do not have to be an Lmod expert to attend. Beginners welcome. The Zoom meeting link is https://utexas.zoom.us/j/2714596735 Best, Robert |
|
From: Robert M. <mc...@ta...> - 2024-11-09 00:58:28
|
This needs to go to the mailing list.
Robert
________________________________
From: Robert McLay <mc...@ta...>
Sent: Friday, November 8, 2024 4:23 PM
To: Thomas Eylenbosch <tho...@pa...>
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
I am unable to reproduce your second issue. As an example, I loaded the following modulefile and it worked fine:
local root = "/a-q/b"
prepend_path{"PATH",root,priority=100}
There is something special about your modulefile example that is causing the issue. Please submit a github issue bug report and use the template script found in bugReport/bug_report_template.sh to create your own script that shows the issue. If we can't reproduce it, we can't fix it.
There is something special about what you are doing that we do not currently test for. It could be the value of the path that you are prepending to. It could be spaces or special character (maybe a UTF-8 character) in "root".
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...>
Sent: Friday, November 8, 2024 9:56 AM
To: Robert McLay <mc...@ta...>
Cc: lmo...@li... <lmo...@li...>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
The root variable also has a value defined in the module file. It contains the path where the specific software is located.
However, I believe the priority=100 parameter is causing the error message.
Best regards
Thomas Eylenbosch
From: Robert McLay <mc...@ta...>
Sent: Friday, November 8, 2024 3:13 PM
To: Thomas Eylenbosch <tho...@pa...>
Cc: lmo...@li...
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
I am looking at your bug reports and I can reproduce bug 1: I see the same thing when I reload make_foo_visible. I don't have a solution yet.
As for your second bug report:
2)
When a modulefile has the following statement in the lua file:
prepend_path{"PATH", root, priority=100}
I’m getting the following error message when using lmod from IS690-hide branch:
Lmod has detected the following error: Error in LocationT:search().
What is variable root? Does it have a value or not?
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Thursday, November 7, 2024 2:35 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Oh yes, my mistake...
This is the correct statement I meant:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all current loaded hidden modules are tracked again, which means foo/invisible_A, ... is also tracked for some reason.
Best regards
Thomas Eylenbosch
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Wednesday, November 6, 2024 6:34 PM
To: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Cc: lmo...@li...<mailto:lmo...@li...>
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
I am confused by your first bug report.
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again
What do you mean by the last statement? Aren't all unhidden modules supposed to tracked?
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Wednesday, November 6, 2024 10:37 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for solving the reported bug.
I have tested your changes with lmod from the IS690-hide branch, and the reported bug is now solved for me.
However, I still see 2 other minor bugs
1 )
When I load the modules in the order below:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again.
2)
When a modulefile has the following statement in the lua file:
prepend_path{"PATH", root, priority=100}
I’m getting the following error message when using lmod from IS690-hide branch:
Lmod has detected the following error: Error in LocationT:search().
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Monday, November 4, 2024 7:09 PM
To: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Cc: lmo...@li...<mailto:lmo...@li...>
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
Thanks very much for the bug report! While your bug seemed small, it had far reaching changes needed to support updating $MODULEPATH and knowing about a hidden module in that new directory in $MODULEPATH. Again, thanks for the report. This allowed us to fixed an issue that we didn't see.
Please test the IS690-hide branch to see if it works for you. Please report back to the mailing list on how it goes for you. This branch is going to be merged in the main before SC'24 (Nov. 19+)
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Tuesday, October 15, 2024 3:19 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for the feedback and providing a fix in Lmod version 8.7.53
It is now working fine.
However, I still see a minor bug with the following use case:
Case 1:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
==> only foo/visible_B is tracked which is normal behaviour
Case 2:
$ module load modules/make_foo_visible
$ module load modules/make_foo_visible foo/invisible_A foo/visible_B
==> Both modules foo/invisible_A foo/visible_B are tracked, which should not be the intention.
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Sunday, October 13, 2024 3:02 AM
To: lmo...@li...<mailto:lmo...@li...>; Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Subject: [EXT] Re: Lmod mrc function not working anymore
If you wish to debug your current code, please read the following new section on how to debug Lmod:
https://lmod.readthedocs.io/en/latest/165_debugging_lmod.html
However, the internals of Lmod can change between versions so relying on internal can lead to problems. In fact, the internals of how the Lmod determines if a module is visible has changed considerably in 8.7.51+
I believe that I have a better way to know if a module is visible or not. As the updated documentation shows:
https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local s_msgT = {}
local function load_hook(t)
-- the arg t is a table:
-- t.modFullName: the module full name: (i.e: gcc/4.7.2)
-- t.fn: The file name: (i.e /apps/modulefiles/Core/gcc/4.7.2.lua)
-- t.mname: The Module Name object.
local isVisible = t.mname:isVisible()
I have created and uploaded a new version of Lmod 8.7.53 which includes the new member function MName:isVisible(). If you have a chance, please test Lmod 8.7.53 to see if works for you.
Best
________________________________
From: Thomas Eylenbosch via Lmod-users <lmo...@li...<mailto:lmo...@li...>>
Sent: Friday, October 11, 2024 4:36 AM
To: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: [Lmod-users] Lmod mrc function not working anymore
Hi Lmod community
We have a customized SitePackage.lua in our platform to track the lmod usage of the users.
Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system:
local mrc = MRC:singleton()
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
What has changed in the Lmod package that now did break this functionality?
I already tried to search related changes in the lmod source code, but I can’t figure it out.
How the SitePackage.lua file looks like:
require("strict")
require("lmod_system_execute")
require("string_utils")
local MRC = require("MRC")
local mrc = MRC:singleton()
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local FrameStk = require("FrameStk")
-- By using the hook.register function, this function "load_hook" is called
-- ever time a module is loaded with the file name and the module name.
local s_msgA = {}
local function isnotempty(s)
return s ~= nil and s ~= ''
end
function load_hook(t)
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
...
end
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
|
|
From: Thomas E. <tho...@pa...> - 2024-11-08 16:29:49
|
Hi Robert
The root variable also has a value defined in the module file. It contains the path where the specific software is located.
However, I believe the priority=100 parameter is causing the error message.
Best regards
Thomas Eylenbosch
From: Robert McLay <mc...@ta...>
Sent: Friday, November 8, 2024 3:13 PM
To: Thomas Eylenbosch <tho...@pa...>
Cc: lmo...@li...
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
I am looking at your bug reports and I can reproduce bug 1: I see the same thing when I reload make_foo_visible. I don't have a solution yet.
As for your second bug report:
2)
When a modulefile has the following statement in the lua file:
prepend_path{"PATH", root, priority=100}
I'm getting the following error message when using lmod from IS690-hide branch:
Lmod has detected the following error: Error in LocationT:search().
What is variable root? Does it have a value or not?
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Thursday, November 7, 2024 2:35 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Oh yes, my mistake...
This is the correct statement I meant:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all current loaded hidden modules are tracked again, which means foo/invisible_A, ... is also tracked for some reason.
Best regards
Thomas Eylenbosch
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Wednesday, November 6, 2024 6:34 PM
To: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Cc: lmo...@li...<mailto:lmo...@li...>
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
I am confused by your first bug report.
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again
What do you mean by the last statement? Aren't all unhidden modules supposed to tracked?
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Wednesday, November 6, 2024 10:37 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for solving the reported bug.
I have tested your changes with lmod from the IS690-hide branch, and the reported bug is now solved for me.
However, I still see 2 other minor bugs
1 )
When I load the modules in the order below:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again.
2)
When a modulefile has the following statement in the lua file:
prepend_path{"PATH", root, priority=100}
I'm getting the following error message when using lmod from IS690-hide branch:
Lmod has detected the following error: Error in LocationT:search().
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Monday, November 4, 2024 7:09 PM
To: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Cc: lmo...@li...<mailto:lmo...@li...>
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
Thanks very much for the bug report! While your bug seemed small, it had far reaching changes needed to support updating $MODULEPATH and knowing about a hidden module in that new directory in $MODULEPATH. Again, thanks for the report. This allowed us to fixed an issue that we didn't see.
Please test the IS690-hide branch to see if it works for you. Please report back to the mailing list on how it goes for you. This branch is going to be merged in the main before SC'24 (Nov. 19+)
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Tuesday, October 15, 2024 3:19 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for the feedback and providing a fix in Lmod version 8.7.53
It is now working fine.
However, I still see a minor bug with the following use case:
Case 1:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
==> only foo/visible_B is tracked which is normal behaviour
Case 2:
$ module load modules/make_foo_visible
$ module load modules/make_foo_visible foo/invisible_A foo/visible_B
==> Both modules foo/invisible_A foo/visible_B are tracked, which should not be the intention.
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Sunday, October 13, 2024 3:02 AM
To: lmo...@li...<mailto:lmo...@li...>; Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Subject: [EXT] Re: Lmod mrc function not working anymore
If you wish to debug your current code, please read the following new section on how to debug Lmod:
https://lmod.readthedocs.io/en/latest/165_debugging_lmod.html
However, the internals of Lmod can change between versions so relying on internal can lead to problems. In fact, the internals of how the Lmod determines if a module is visible has changed considerably in 8.7.51+
I believe that I have a better way to know if a module is visible or not. As the updated documentation shows:
https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local s_msgT = {}
local function load_hook(t)
-- the arg t is a table:
-- t.modFullName: the module full name: (i.e: gcc/4.7.2)
-- t.fn: The file name: (i.e /apps/modulefiles/Core/gcc/4.7.2.lua)
-- t.mname: The Module Name object.
local isVisible = t.mname:isVisible()
I have created and uploaded a new version of Lmod 8.7.53 which includes the new member function MName:isVisible(). If you have a chance, please test Lmod 8.7.53 to see if works for you.
Best
________________________________
From: Thomas Eylenbosch via Lmod-users <lmo...@li...<mailto:lmo...@li...>>
Sent: Friday, October 11, 2024 4:36 AM
To: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: [Lmod-users] Lmod mrc function not working anymore
Hi Lmod community
We have a customized SitePackage.lua in our platform to track the lmod usage of the users.
Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system:
local mrc = MRC:singleton()
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
What has changed in the Lmod package that now did break this functionality?
I already tried to search related changes in the lmod source code, but I can't figure it out.
How the SitePackage.lua file looks like:
require("strict")
require("lmod_system_execute")
require("string_utils")
local MRC = require("MRC")
local mrc = MRC:singleton()
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local FrameStk = require("FrameStk")
-- By using the hook.register function, this function "load_hook" is called
-- ever time a module is loaded with the file name and the module name.
local s_msgA = {}
local function isnotempty(s)
return s ~= nil and s ~= ''
end
function load_hook(t)
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
...
end
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
|
|
From: Robert M. <mc...@ta...> - 2024-11-08 14:13:04
|
I am looking at your bug reports and I can reproduce bug 1: I see the same thing when I reload make_foo_visible. I don't have a solution yet.
As for your second bug report:
2)
When a modulefile has the following statement in the lua file:
prepend_path{"PATH", root, priority=100}
I’m getting the following error message when using lmod from IS690-hide branch:
Lmod has detected the following error: Error in LocationT:search().
What is variable root? Does it have a value or not?
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...>
Sent: Thursday, November 7, 2024 2:35 AM
To: Robert McLay <mc...@ta...>
Cc: lmo...@li... <lmo...@li...>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Oh yes, my mistake...
This is the correct statement I meant:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all current loaded hidden modules are tracked again, which means foo/invisible_A, ... is also tracked for some reason.
Best regards
Thomas Eylenbosch
From: Robert McLay <mc...@ta...>
Sent: Wednesday, November 6, 2024 6:34 PM
To: Thomas Eylenbosch <tho...@pa...>
Cc: lmo...@li...
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
I am confused by your first bug report.
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again
What do you mean by the last statement? Aren't all unhidden modules supposed to tracked?
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Wednesday, November 6, 2024 10:37 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for solving the reported bug.
I have tested your changes with lmod from the IS690-hide branch, and the reported bug is now solved for me.
However, I still see 2 other minor bugs
1 )
When I load the modules in the order below:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again.
2)
When a modulefile has the following statement in the lua file:
prepend_path{"PATH", root, priority=100}
I’m getting the following error message when using lmod from IS690-hide branch:
Lmod has detected the following error: Error in LocationT:search().
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Monday, November 4, 2024 7:09 PM
To: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Cc: lmo...@li...<mailto:lmo...@li...>
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
Thanks very much for the bug report! While your bug seemed small, it had far reaching changes needed to support updating $MODULEPATH and knowing about a hidden module in that new directory in $MODULEPATH. Again, thanks for the report. This allowed us to fixed an issue that we didn't see.
Please test the IS690-hide branch to see if it works for you. Please report back to the mailing list on how it goes for you. This branch is going to be merged in the main before SC'24 (Nov. 19+)
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Tuesday, October 15, 2024 3:19 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for the feedback and providing a fix in Lmod version 8.7.53
It is now working fine.
However, I still see a minor bug with the following use case:
Case 1:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
==> only foo/visible_B is tracked which is normal behaviour
Case 2:
$ module load modules/make_foo_visible
$ module load modules/make_foo_visible foo/invisible_A foo/visible_B
==> Both modules foo/invisible_A foo/visible_B are tracked, which should not be the intention.
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Sunday, October 13, 2024 3:02 AM
To: lmo...@li...<mailto:lmo...@li...>; Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Subject: [EXT] Re: Lmod mrc function not working anymore
If you wish to debug your current code, please read the following new section on how to debug Lmod:
https://lmod.readthedocs.io/en/latest/165_debugging_lmod.html
However, the internals of Lmod can change between versions so relying on internal can lead to problems. In fact, the internals of how the Lmod determines if a module is visible has changed considerably in 8.7.51+
I believe that I have a better way to know if a module is visible or not. As the updated documentation shows:
https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local s_msgT = {}
local function load_hook(t)
-- the arg t is a table:
-- t.modFullName: the module full name: (i.e: gcc/4.7.2)
-- t.fn: The file name: (i.e /apps/modulefiles/Core/gcc/4.7.2.lua)
-- t.mname: The Module Name object.
local isVisible = t.mname:isVisible()
I have created and uploaded a new version of Lmod 8.7.53 which includes the new member function MName:isVisible(). If you have a chance, please test Lmod 8.7.53 to see if works for you.
Best
________________________________
From: Thomas Eylenbosch via Lmod-users <lmo...@li...<mailto:lmo...@li...>>
Sent: Friday, October 11, 2024 4:36 AM
To: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: [Lmod-users] Lmod mrc function not working anymore
Hi Lmod community
We have a customized SitePackage.lua in our platform to track the lmod usage of the users.
Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system:
local mrc = MRC:singleton()
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
What has changed in the Lmod package that now did break this functionality?
I already tried to search related changes in the lmod source code, but I can’t figure it out.
How the SitePackage.lua file looks like:
require("strict")
require("lmod_system_execute")
require("string_utils")
local MRC = require("MRC")
local mrc = MRC:singleton()
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local FrameStk = require("FrameStk")
-- By using the hook.register function, this function "load_hook" is called
-- ever time a module is loaded with the file name and the module name.
local s_msgA = {}
local function isnotempty(s)
return s ~= nil and s ~= ''
end
function load_hook(t)
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
...
end
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
|
|
From: Thomas E. <tho...@pa...> - 2024-11-07 09:09:35
|
Oh yes, my mistake...
This is the correct statement I meant:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all current loaded hidden modules are tracked again, which means foo/invisible_A, ... is also tracked for some reason.
Best regards
Thomas Eylenbosch
From: Robert McLay <mc...@ta...>
Sent: Wednesday, November 6, 2024 6:34 PM
To: Thomas Eylenbosch <tho...@pa...>
Cc: lmo...@li...
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
I am confused by your first bug report.
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again
What do you mean by the last statement? Aren't all unhidden modules supposed to tracked?
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Wednesday, November 6, 2024 10:37 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for solving the reported bug.
I have tested your changes with lmod from the IS690-hide branch, and the reported bug is now solved for me.
However, I still see 2 other minor bugs
1 )
When I load the modules in the order below:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again.
2)
When a modulefile has the following statement in the lua file:
prepend_path{"PATH", root, priority=100}
I'm getting the following error message when using lmod from IS690-hide branch:
Lmod has detected the following error: Error in LocationT:search().
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Monday, November 4, 2024 7:09 PM
To: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Cc: lmo...@li...<mailto:lmo...@li...>
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
Thanks very much for the bug report! While your bug seemed small, it had far reaching changes needed to support updating $MODULEPATH and knowing about a hidden module in that new directory in $MODULEPATH. Again, thanks for the report. This allowed us to fixed an issue that we didn't see.
Please test the IS690-hide branch to see if it works for you. Please report back to the mailing list on how it goes for you. This branch is going to be merged in the main before SC'24 (Nov. 19+)
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Tuesday, October 15, 2024 3:19 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for the feedback and providing a fix in Lmod version 8.7.53
It is now working fine.
However, I still see a minor bug with the following use case:
Case 1:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
==> only foo/visible_B is tracked which is normal behaviour
Case 2:
$ module load modules/make_foo_visible
$ module load modules/make_foo_visible foo/invisible_A foo/visible_B
==> Both modules foo/invisible_A foo/visible_B are tracked, which should not be the intention.
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Sunday, October 13, 2024 3:02 AM
To: lmo...@li...<mailto:lmo...@li...>; Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Subject: [EXT] Re: Lmod mrc function not working anymore
If you wish to debug your current code, please read the following new section on how to debug Lmod:
https://lmod.readthedocs.io/en/latest/165_debugging_lmod.html
However, the internals of Lmod can change between versions so relying on internal can lead to problems. In fact, the internals of how the Lmod determines if a module is visible has changed considerably in 8.7.51+
I believe that I have a better way to know if a module is visible or not. As the updated documentation shows:
https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local s_msgT = {}
local function load_hook(t)
-- the arg t is a table:
-- t.modFullName: the module full name: (i.e: gcc/4.7.2)
-- t.fn: The file name: (i.e /apps/modulefiles/Core/gcc/4.7.2.lua)
-- t.mname: The Module Name object.
local isVisible = t.mname:isVisible()
I have created and uploaded a new version of Lmod 8.7.53 which includes the new member function MName:isVisible(). If you have a chance, please test Lmod 8.7.53 to see if works for you.
Best
________________________________
From: Thomas Eylenbosch via Lmod-users <lmo...@li...<mailto:lmo...@li...>>
Sent: Friday, October 11, 2024 4:36 AM
To: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: [Lmod-users] Lmod mrc function not working anymore
Hi Lmod community
We have a customized SitePackage.lua in our platform to track the lmod usage of the users.
Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system:
local mrc = MRC:singleton()
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
What has changed in the Lmod package that now did break this functionality?
I already tried to search related changes in the lmod source code, but I can't figure it out.
How the SitePackage.lua file looks like:
require("strict")
require("lmod_system_execute")
require("string_utils")
local MRC = require("MRC")
local mrc = MRC:singleton()
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local FrameStk = require("FrameStk")
-- By using the hook.register function, this function "load_hook" is called
-- ever time a module is loaded with the file name and the module name.
local s_msgA = {}
local function isnotempty(s)
return s ~= nil and s ~= ''
end
function load_hook(t)
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
...
end
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
|
|
From: Thomas E. <tho...@pa...> - 2024-11-06 18:10:58
|
Hi Robert
Thank you for solving the reported bug.
I have tested your changes with lmod from the IS690-hide branch, and the reported bug is now solved for me.
However, I still see 2 other minor bugs
1 )
When I load the modules in the order below:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again.
2)
When a modulefile has the following statement in the lua file:
prepend_path{"PATH", root, priority=100}
I'm getting the following error message when using lmod from IS690-hide branch:
Lmod has detected the following error: Error in LocationT:search().
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...>
Sent: Monday, November 4, 2024 7:09 PM
To: Thomas Eylenbosch <tho...@pa...>
Cc: lmo...@li...
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
Thanks very much for the bug report! While your bug seemed small, it had far reaching changes needed to support updating $MODULEPATH and knowing about a hidden module in that new directory in $MODULEPATH. Again, thanks for the report. This allowed us to fixed an issue that we didn't see.
Please test the IS690-hide branch to see if it works for you. Please report back to the mailing list on how it goes for you. This branch is going to be merged in the main before SC'24 (Nov. 19+)
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Tuesday, October 15, 2024 3:19 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for the feedback and providing a fix in Lmod version 8.7.53
It is now working fine.
However, I still see a minor bug with the following use case:
Case 1:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
==> only foo/visible_B is tracked which is normal behaviour
Case 2:
$ module load modules/make_foo_visible
$ module load modules/make_foo_visible foo/invisible_A foo/visible_B
==> Both modules foo/invisible_A foo/visible_B are tracked, which should not be the intention.
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Sunday, October 13, 2024 3:02 AM
To: lmo...@li...<mailto:lmo...@li...>; Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Subject: [EXT] Re: Lmod mrc function not working anymore
If you wish to debug your current code, please read the following new section on how to debug Lmod:
https://lmod.readthedocs.io/en/latest/165_debugging_lmod.html
However, the internals of Lmod can change between versions so relying on internal can lead to problems. In fact, the internals of how the Lmod determines if a module is visible has changed considerably in 8.7.51+
I believe that I have a better way to know if a module is visible or not. As the updated documentation shows:
https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local s_msgT = {}
local function load_hook(t)
-- the arg t is a table:
-- t.modFullName: the module full name: (i.e: gcc/4.7.2)
-- t.fn: The file name: (i.e /apps/modulefiles/Core/gcc/4.7.2.lua)
-- t.mname: The Module Name object.
local isVisible = t.mname:isVisible()
I have created and uploaded a new version of Lmod 8.7.53 which includes the new member function MName:isVisible(). If you have a chance, please test Lmod 8.7.53 to see if works for you.
Best
________________________________
From: Thomas Eylenbosch via Lmod-users <lmo...@li...<mailto:lmo...@li...>>
Sent: Friday, October 11, 2024 4:36 AM
To: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: [Lmod-users] Lmod mrc function not working anymore
Hi Lmod community
We have a customized SitePackage.lua in our platform to track the lmod usage of the users.
Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system:
local mrc = MRC:singleton()
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
What has changed in the Lmod package that now did break this functionality?
I already tried to search related changes in the lmod source code, but I can't figure it out.
How the SitePackage.lua file looks like:
require("strict")
require("lmod_system_execute")
require("string_utils")
local MRC = require("MRC")
local mrc = MRC:singleton()
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local FrameStk = require("FrameStk")
-- By using the hook.register function, this function "load_hook" is called
-- ever time a module is loaded with the file name and the module name.
local s_msgA = {}
local function isnotempty(s)
return s ~= nil and s ~= ''
end
function load_hook(t)
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
...
end
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
|
|
From: Robert M. <mc...@ta...> - 2024-11-06 17:34:32
|
I am confused by your first bug report.
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again
What do you mean by the last statement? Aren't all unhidden modules supposed to tracked?
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...>
Sent: Wednesday, November 6, 2024 10:37 AM
To: Robert McLay <mc...@ta...>
Cc: lmo...@li... <lmo...@li...>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for solving the reported bug.
I have tested your changes with lmod from the IS690-hide branch, and the reported bug is now solved for me.
However, I still see 2 other minor bugs
1 )
When I load the modules in the order below:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
$ module load modules/make_foo_visible # => all unhidden modules are tracked again.
2)
When a modulefile has the following statement in the lua file:
prepend_path{"PATH", root, priority=100}
I’m getting the following error message when using lmod from IS690-hide branch:
Lmod has detected the following error: Error in LocationT:search().
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...>
Sent: Monday, November 4, 2024 7:09 PM
To: Thomas Eylenbosch <tho...@pa...>
Cc: lmo...@li...
Subject: Re: [EXT] Re: Lmod mrc function not working anymore
Thanks very much for the bug report! While your bug seemed small, it had far reaching changes needed to support updating $MODULEPATH and knowing about a hidden module in that new directory in $MODULEPATH. Again, thanks for the report. This allowed us to fixed an issue that we didn't see.
Please test the IS690-hide branch to see if it works for you. Please report back to the mailing list on how it goes for you. This branch is going to be merged in the main before SC'24 (Nov. 19+)
Best,
Robert
________________________________
From: Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Sent: Tuesday, October 15, 2024 3:19 AM
To: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Cc: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: RE: [EXT] Re: Lmod mrc function not working anymore
Hi Robert
Thank you for the feedback and providing a fix in Lmod version 8.7.53
It is now working fine.
However, I still see a minor bug with the following use case:
Case 1:
$ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules
$ module load foo/invisible_A foo/visible_B
==> only foo/visible_B is tracked which is normal behaviour
Case 2:
$ module load modules/make_foo_visible
$ module load modules/make_foo_visible foo/invisible_A foo/visible_B
==> Both modules foo/invisible_A foo/visible_B are tracked, which should not be the intention.
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
From: Robert McLay <mc...@ta...<mailto:mc...@ta...>>
Sent: Sunday, October 13, 2024 3:02 AM
To: lmo...@li...<mailto:lmo...@li...>; Thomas Eylenbosch <tho...@pa...<mailto:tho...@pa...>>
Subject: [EXT] Re: Lmod mrc function not working anymore
If you wish to debug your current code, please read the following new section on how to debug Lmod:
https://lmod.readthedocs.io/en/latest/165_debugging_lmod.html
However, the internals of Lmod can change between versions so relying on internal can lead to problems. In fact, the internals of how the Lmod determines if a module is visible has changed considerably in 8.7.51+
I believe that I have a better way to know if a module is visible or not. As the updated documentation shows:
https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local s_msgT = {}
local function load_hook(t)
-- the arg t is a table:
-- t.modFullName: the module full name: (i.e: gcc/4.7.2)
-- t.fn: The file name: (i.e /apps/modulefiles/Core/gcc/4.7.2.lua)
-- t.mname: The Module Name object.
local isVisible = t.mname:isVisible()
I have created and uploaded a new version of Lmod 8.7.53 which includes the new member function MName:isVisible(). If you have a chance, please test Lmod 8.7.53 to see if works for you.
Best
________________________________
From: Thomas Eylenbosch via Lmod-users <lmo...@li...<mailto:lmo...@li...>>
Sent: Friday, October 11, 2024 4:36 AM
To: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>>
Subject: [Lmod-users] Lmod mrc function not working anymore
Hi Lmod community
We have a customized SitePackage.lua in our platform to track the lmod usage of the users.
Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system:
local mrc = MRC:singleton()
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
What has changed in the Lmod package that now did break this functionality?
I already tried to search related changes in the lmod source code, but I can’t figure it out.
How the SitePackage.lua file looks like:
require("strict")
require("lmod_system_execute")
require("string_utils")
local MRC = require("MRC")
local mrc = MRC:singleton()
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local FrameStk = require("FrameStk")
-- By using the hook.register function, this function "load_hook" is called
-- ever time a module is loaded with the file name and the module name.
local s_msgA = {}
local function isnotempty(s)
return s ~= nil and s ~= ''
end
function load_hook(t)
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
...
end
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be/>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
|
|
From: Robert M. <mc...@ta...> - 2024-11-04 18:09:43
|
Thanks very much for the bug report! While your bug seemed small, it had far reaching changes needed to support updating $MODULEPATH and knowing about a hidden module in that new directory in $MODULEPATH. Again, thanks for the report. This allowed us to fixed an issue that we didn't see. Please test the IS690-hide branch to see if it works for you. Please report back to the mailing list on how it goes for you. This branch is going to be merged in the main before SC'24 (Nov. 19+) Best, Robert ________________________________ From: Thomas Eylenbosch <tho...@pa...> Sent: Tuesday, October 15, 2024 3:19 AM To: Robert McLay <mc...@ta...> Cc: lmo...@li... <lmo...@li...> Subject: RE: [EXT] Re: Lmod mrc function not working anymore Hi Robert Thank you for the feedback and providing a fix in Lmod version 8.7.53 It is now working fine. However, I still see a minor bug with the following use case: Case 1: $ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules $ module load foo/invisible_A foo/visible_B ==> only foo/visible_B is tracked which is normal behaviour Case 2: $ module load modules/make_foo_visible $ module load modules/make_foo_visible foo/invisible_A foo/visible_B ==> Both modules foo/invisible_A foo/visible_B are tracked, which should not be the intention. Best regards / met vriendelijke groeten Thomas Eylenbosch DevOps Engineer, Gluo N.V. Email: tho...@pa...<mailto:tho...@pa...> Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium BASF Belgium Coordination Center CommV Scheldelaan 600, 2040 Antwerpen, België RPR Antwerpen (afd. Antwerpen) BTW BE0862.390.376 www.basf.be<http://www.basf.be/> Deutsche Bank AG IBAN: BE43 8262 8044 4801 BIC: DEUTBEBE Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html From: Robert McLay <mc...@ta...> Sent: Sunday, October 13, 2024 3:02 AM To: lmo...@li...; Thomas Eylenbosch <tho...@pa...> Subject: [EXT] Re: Lmod mrc function not working anymore If you wish to debug your current code, please read the following new section on how to debug Lmod: https://lmod.readthedocs.io/en/latest/165_debugging_lmod.html However, the internals of Lmod can change between versions so relying on internal can lead to problems. In fact, the internals of how the Lmod determines if a module is visible has changed considerably in 8.7.51+ I believe that I have a better way to know if a module is visible or not. As the updated documentation shows: https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html local hook = require("Hook") local uname = require("posix").uname local cosmic = require("Cosmic"):singleton() local syshost = cosmic:value("LMOD_SYSHOST") local s_msgT = {} local function load_hook(t) -- the arg t is a table: -- t.modFullName: the module full name: (i.e: gcc/4.7.2) -- t.fn: The file name: (i.e /apps/modulefiles/Core/gcc/4.7.2.lua) -- t.mname: The Module Name object. local isVisible = t.mname:isVisible() I have created and uploaded a new version of Lmod 8.7.53 which includes the new member function MName:isVisible(). If you have a chance, please test Lmod 8.7.53 to see if works for you. Best ________________________________ From: Thomas Eylenbosch via Lmod-users <lmo...@li...<mailto:lmo...@li...>> Sent: Friday, October 11, 2024 4:36 AM To: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>> Subject: [Lmod-users] Lmod mrc function not working anymore Hi Lmod community We have a customized SitePackage.lua in our platform to track the lmod usage of the users. Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system: local mrc = MRC:singleton() local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn}) What has changed in the Lmod package that now did break this functionality? I already tried to search related changes in the lmod source code, but I can’t figure it out. How the SitePackage.lua file looks like: require("strict") require("lmod_system_execute") require("string_utils") local MRC = require("MRC") local mrc = MRC:singleton() local hook = require("Hook") local uname = require("posix").uname local cosmic = require("Cosmic"):singleton() local syshost = cosmic:value("LMOD_SYSHOST") local FrameStk = require("FrameStk") -- By using the hook.register function, this function "load_hook" is called -- ever time a module is loaded with the file name and the module name. local s_msgA = {} local function isnotempty(s) return s ~= nil and s ~= '' end function load_hook(t) local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn}) ... end Best regards / met vriendelijke groeten Thomas Eylenbosch DevOps Engineer, Gluo N.V. Email: tho...@pa...<mailto:tho...@pa...> Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium BASF Belgium Coordination Center CommV Scheldelaan 600, 2040 Antwerpen, België RPR Antwerpen (afd. Antwerpen) BTW BE0862.390.376 www.basf.be<http://www.basf.be/> Deutsche Bank AG IBAN: BE43 8262 8044 4801 BIC: DEUTBEBE Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html |
|
From: Robert M. <mc...@ta...> - 2024-11-01 16:52:01
|
Matthew and I have unavoidable conflicts on Nov. 5th. So, the monthly zoom meeting will be on November 12th at 9:30 US Central. There are many things to discuss at the meeting. Among them are: * Q/A * Support for TCL 9.0 (Kinda) * Support for non-reversible functions like: setenv, load, etc. * Support for controlling hidden module display with env var. LMOD_SHOW_HIDDEN * Bug fixes for how hidden modules are handled when $MODULEPATH changes. * Show and Tell at SC'24 You do not have to be an Lmod expert to attend. Beginners welcome. The Zoom meeting link is https://utexas.zoom.us/j/2714596735 Join our Cloud HD Video Meeting<https://utexas.zoom.us/j/2714596735> Zoom is the leader in modern enterprise video communications, with an easy, reliable cloud platform for video and audio conferencing, chat, and webinars across mobile, desktop, and room systems. Zoom Rooms is the original software-based conference room solution used around the world in board, conference, huddle, and training rooms, as well as executive offices and classrooms. Founded in 2011, Zoom helps businesses and organizations bring their teams together in a frictionless environment to get more done. Zoom is a publicly traded company headquartered in San Jose, CA. utexas.zoom.us Best, Robert |
|
From: Raghu R. - N. A. <rag...@no...> - 2024-10-30 16:03:25
|
Hi Robert, We are using Lmod/8.7.32. I am including the actual error message from the user so that you have the full context, but information in the included link has sufficient detail: <quote> The lmod issue happened again in a recent PR: https://gist.github.com/emcbot/bc858b317c351bc822c1d1381f7f6091 Every one of our jobs calls this section of code, so it was exercised hundreds of time in these tests for this PR without having a problem. </quote> Based on what I am seeing it looks like somehow MODULEPATH under some circumstances is getting undefined or not getting set! They are using the "module reset" command and that is something that seems to be used less frequently in our environment. We have not seen any other reports of this issue, so very likely related to "reset". Clearly they run this all the time and seems to happen only sometimes. Any thoughts on how to troubleshoot this or what may be causing the issue? Very much appreciate any guidance you can provide! -- Raghu Reddy, RDHPCS Support Team GDIT contractor at NOAA/NWS/NCEP/EMC 5830 University Research Court, Suite 2146 College Park, MD 20740 Phone: 301-683-3771 (o), 301-683-3703 (fax), 412-439-8741 (c) |
|
From: Stefan M. <Ste...@DL...> - 2024-10-22 06:34:31
|
Hi Robert, thank you for the hints & ideas. The container idea looks promising at a first look - but i assume using this on a cluster-enviroment it unlocks much more problem as it solves ;-) So number 2-4 seems to be better options - in that order. Will play with them to see what pros and cons they have. Currently i follow another idea by some shell-scripting & python: o diff the environment variables before / after loading the modules you need for an executable o extract all paths which points to libs & includes from the diff o copy them all (including subdirs) to another (user-)directory to hold the snapshot of the libs o change all paths to the new directory in the environment variables o use the new environment for compiling and running the executeables This works already - but found that spack bakes the "old" paths with rpath into the libs - so if you lookup the executeable with ldd you will find them still there. The next step is then change them with chrpath... Best regards and thanks again for your ideas, Stefan > There is no easy way to do this. All the ones that I can think of are > painful. For most of the following ideas assume that a module version > does not change. So, while package A might change. The package > pointed to by A/3.2.1 does not or rarely does. Your idea of copying > modules to a private location is possible, however you might have edit > many SPACK modules to know about where you have copied the modules. > > 1. > Make a container. You copy everything you need into the container > and you don't have to worry about anything changing on you. You > will never have to worry about even A/3.2.1 changing. > 2. > Outside of a container. You can set LMOD_EXACT_MATCH. This way > you have to set every version for every module. I do not know how > this will work with spack generated modules. > 3. > You can create a LMOD_MODULERC file to specify (and override) what > the system defaults are. This ought to work with SPACK. > 4. > Finally, you might consider installing XALT. This is another > project I'm responsible for. See xalt.readthedocs.io for details. > The advantage here with XALT is that you can have the current set > of modules recorded in the executable. You don't have to record > what gets run (Which is the main point of XALT) but knowing what > modules were loaded when built might be helpful > > For example here is the output from a simple hello world program: > > % xalt_extract_record ./hello > **************************************** > XALT Watermark: ./hello > **************************************** > Build_CWD /path/to/t/xalt > Build_Epoch 1636646158.6015 > Build_LMFILES ... > Build_LOADEDMODULES > intel/19.1.1:impi/19.0.9:python3/3.7.0:cmake/3.20.3:pmix/3.1.4:hwloc/1.11.12:TACC:noweb/2.11b:autotools/1.2:phdf5/1.10.4:boost/1.72:git/2.24.1:ddt/20.0.2:lmod:settarg:xalt/2.10.34 > Build_OS Linux 3.10.0-1160.45.1.el7.x86_64 > Build_Syshost frontera > Build_UUID eb0405e5-e136-44ce-b06f-342bc504e1db > Build_User mclay > Build_Year 2021 > Build_compiler gcc > Build_compilerPath /opt/apps/gcc/8.3.0/bin/gcc > Build_date Thu Nov 11 09:55:58 2021 > Build_host c205-036.frontera.tacc.utexas.edu > XALT_Version 2.10.34 > > Best, > Robert. > ------------------------------------------------------------------------ > *From:* Stefan Melber via Lmod-users <lmo...@li...> > *Sent:* Sunday, October 20, 2024 12:22 AM > *To:* lmod <lmo...@li...> > *Subject:* [Lmod-users] Freeze status of modules > Hi, > > > i have a newbie question using lmod: i use lmod on a cluster with huge > number of modules (compiled with spack) to create an environment for > compiling a cfd code with many dependencies. However the module > environment changes on a daily basis because of users requests and > security updates and sometimes hit modules i am using for the > compilation. So there is always the risk the binaries will not run > anymore at some point in time. Of course we tried together with the > admins to get some parts of the modules "kind of stable" - but in real > live thats not so easy. > > So the idea is to "freeze" at some point in time a set of modules and > compile against them to be sure the dependent binaries will work at any > time. The question is now: are the some ideas to do that with lmod or > some other tools? > > > Some ideas are: > > o load the needed modules --> identify the directories with the libs & > enviroment variables --> create a local copy --> change the path to the > local copy > > o "Personal Hierarchy Mirroring the System Hierarchy" like > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flmod.readthedocs.io%2Fen%2Flatest%2F340_inherit.html&data=05%7C02%7C%7C9c0dc7b21f0c46a9d51608dcf0c97d25%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C638649995418832643%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=UIVOBCdNldU86KyCOLS7q6lGAATeQstKzkJcWQ%2FFsqI%3D&reserved=0 > <https://lmod.readthedocs.io/en/latest/340_inherit.html> - but i am not > sure how to duplicate a list of given modules > > > Any suggestions are welcome ;-) > > Best regards and thank you in advance, > > > Stefan > > > > > > > _______________________________________________ > Lmod-users mailing list > Lmo...@li... > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Flmod-users&data=05%7C02%7C%7C9c0dc7b21f0c46a9d51608dcf0c97d25%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C638649995418849130%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=kZik9DVRLhGCKJ8cCzZ%2B95MjgvcqUsxtHbbX5blg4fc%3D&reserved=0 > <https://lists.sourceforge.net/lists/listinfo/lmod-users> > >> This message is from an external sender. Learn more about why this << > >> matters at https://links.utexas.edu/rtyclf. << |
|
From: Robert M. <mc...@ta...> - 2024-10-21 15:41:57
|
There is no easy way to do this. All the ones that I can think of are painful. For most of the following ideas assume that a module version does not change. So, while package A might change. The package pointed to by A/3.2.1 does not or rarely does. Your idea of copying modules to a private location is possible, however you might have edit many SPACK modules to know about where you have copied the modules. 1. Make a container. You copy everything you need into the container and you don't have to worry about anything changing on you. You will never have to worry about even A/3.2.1 changing. 2. Outside of a container. You can set LMOD_EXACT_MATCH. This way you have to set every version for every module. I do not know how this will work with spack generated modules. 3. You can create a LMOD_MODULERC file to specify (and override) what the system defaults are. This ought to work with SPACK. 4. Finally, you might consider installing XALT. This is another project I'm responsible for. See xalt.readthedocs.io for details. The advantage here with XALT is that you can have the current set of modules recorded in the executable. You don't have to record what gets run (Which is the main point of XALT) but knowing what modules were loaded when built might be helpful For example here is the output from a simple hello world program: % xalt_extract_record ./hello **************************************** XALT Watermark: ./hello **************************************** Build_CWD /path/to/t/xalt Build_Epoch 1636646158.6015 Build_LMFILES ... Build_LOADEDMODULES intel/19.1.1:impi/19.0.9:python3/3.7.0:cmake/3.20.3:pmix/3.1.4:hwloc/1.11.12:TACC:noweb/2.11b:autotools/1.2:phdf5/1.10.4:boost/1.72:git/2.24.1:ddt/20.0.2:lmod:settarg:xalt/2.10.34 Build_OS Linux 3.10.0-1160.45.1.el7.x86_64 Build_Syshost frontera Build_UUID eb0405e5-e136-44ce-b06f-342bc504e1db Build_User mclay Build_Year 2021 Build_compiler gcc Build_compilerPath /opt/apps/gcc/8.3.0/bin/gcc Build_date Thu Nov 11 09:55:58 2021 Build_host c205-036.frontera.tacc.utexas.edu XALT_Version 2.10.34 Best, Robert. ________________________________ From: Stefan Melber via Lmod-users <lmo...@li...> Sent: Sunday, October 20, 2024 12:22 AM To: lmod <lmo...@li...> Subject: [Lmod-users] Freeze status of modules Hi, i have a newbie question using lmod: i use lmod on a cluster with huge number of modules (compiled with spack) to create an environment for compiling a cfd code with many dependencies. However the module environment changes on a daily basis because of users requests and security updates and sometimes hit modules i am using for the compilation. So there is always the risk the binaries will not run anymore at some point in time. Of course we tried together with the admins to get some parts of the modules "kind of stable" - but in real live thats not so easy. So the idea is to "freeze" at some point in time a set of modules and compile against them to be sure the dependent binaries will work at any time. The question is now: are the some ideas to do that with lmod or some other tools? Some ideas are: o load the needed modules --> identify the directories with the libs & enviroment variables --> create a local copy --> change the path to the local copy o "Personal Hierarchy Mirroring the System Hierarchy" like https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flmod.readthedocs.io%2Fen%2Flatest%2F340_inherit.html&data=05%7C02%7C%7C9c0dc7b21f0c46a9d51608dcf0c97d25%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C638649995418832643%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=UIVOBCdNldU86KyCOLS7q6lGAATeQstKzkJcWQ%2FFsqI%3D&reserved=0<https://lmod.readthedocs.io/en/latest/340_inherit.html> - but i am not sure how to duplicate a list of given modules Any suggestions are welcome ;-) Best regards and thank you in advance, Stefan _______________________________________________ Lmod-users mailing list Lmo...@li... https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Flmod-users&data=05%7C02%7C%7C9c0dc7b21f0c46a9d51608dcf0c97d25%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C638649995418849130%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=kZik9DVRLhGCKJ8cCzZ%2B95MjgvcqUsxtHbbX5blg4fc%3D&reserved=0<https://lists.sourceforge.net/lists/listinfo/lmod-users> >> This message is from an external sender. Learn more about why this << >> matters at https://links.utexas.edu/rtyclf. << |
|
From: Stefan M. <Ste...@DL...> - 2024-10-20 05:38:13
|
Hi, i have a newbie question using lmod: i use lmod on a cluster with huge number of modules (compiled with spack) to create an environment for compiling a cfd code with many dependencies. However the module environment changes on a daily basis because of users requests and security updates and sometimes hit modules i am using for the compilation. So there is always the risk the binaries will not run anymore at some point in time. Of course we tried together with the admins to get some parts of the modules "kind of stable" - but in real live thats not so easy. So the idea is to "freeze" at some point in time a set of modules and compile against them to be sure the dependent binaries will work at any time. The question is now: are the some ideas to do that with lmod or some other tools? Some ideas are: o load the needed modules --> identify the directories with the libs & enviroment variables --> create a local copy --> change the path to the local copy o "Personal Hierarchy Mirroring the System Hierarchy" like https://lmod.readthedocs.io/en/latest/340_inherit.html - but i am not sure how to duplicate a list of given modules Any suggestions are welcome ;-) Best regards and thank you in advance, Stefan |
|
From: Thomas E. <tho...@pa...> - 2024-10-15 09:52:31
|
Hi Robert Thank you for the feedback and providing a fix in Lmod version 8.7.53 It is now working fine. However, I still see a minor bug with the following use case: Case 1: $ module load modules/make_foo_visible #extend MODULEPATH to enable loading foo modules $ module load foo/invisible_A foo/visible_B ==> only foo/visible_B is tracked which is normal behaviour Case 2: $ module load modules/make_foo_visible $ module load modules/make_foo_visible foo/invisible_A foo/visible_B ==> Both modules foo/invisible_A foo/visible_B are tracked, which should not be the intention. Best regards / met vriendelijke groeten Thomas Eylenbosch DevOps Engineer, Gluo N.V. Email: tho...@pa...<mailto:tho...@pa...> Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium BASF Belgium Coordination Center CommV Scheldelaan 600, 2040 Antwerpen, België RPR Antwerpen (afd. Antwerpen) BTW BE0862.390.376 www.basf.be<http://www.basf.be/> Deutsche Bank AG IBAN: BE43 8262 8044 4801 BIC: DEUTBEBE Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html From: Robert McLay <mc...@ta...> Sent: Sunday, October 13, 2024 3:02 AM To: lmo...@li...; Thomas Eylenbosch <tho...@pa...> Subject: [EXT] Re: Lmod mrc function not working anymore If you wish to debug your current code, please read the following new section on how to debug Lmod: https://lmod.readthedocs.io/en/latest/165_debugging_lmod.html However, the internals of Lmod can change between versions so relying on internal can lead to problems. In fact, the internals of how the Lmod determines if a module is visible has changed considerably in 8.7.51+ I believe that I have a better way to know if a module is visible or not. As the updated documentation shows: https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html local hook = require("Hook") local uname = require("posix").uname local cosmic = require("Cosmic"):singleton() local syshost = cosmic:value("LMOD_SYSHOST") local s_msgT = {} local function load_hook(t) -- the arg t is a table: -- t.modFullName: the module full name: (i.e: gcc/4.7.2) -- t.fn: The file name: (i.e /apps/modulefiles/Core/gcc/4.7.2.lua) -- t.mname: The Module Name object. local isVisible = t.mname:isVisible() I have created and uploaded a new version of Lmod 8.7.53 which includes the new member function MName:isVisible(). If you have a chance, please test Lmod 8.7.53 to see if works for you. Best ________________________________ From: Thomas Eylenbosch via Lmod-users <lmo...@li...<mailto:lmo...@li...>> Sent: Friday, October 11, 2024 4:36 AM To: lmo...@li...<mailto:lmo...@li...> <lmo...@li...<mailto:lmo...@li...>> Subject: [Lmod-users] Lmod mrc function not working anymore Hi Lmod community We have a customized SitePackage.lua in our platform to track the lmod usage of the users. Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system: local mrc = MRC:singleton() local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn}) What has changed in the Lmod package that now did break this functionality? I already tried to search related changes in the lmod source code, but I can't figure it out. How the SitePackage.lua file looks like: require("strict") require("lmod_system_execute") require("string_utils") local MRC = require("MRC") local mrc = MRC:singleton() local hook = require("Hook") local uname = require("posix").uname local cosmic = require("Cosmic"):singleton() local syshost = cosmic:value("LMOD_SYSHOST") local FrameStk = require("FrameStk") -- By using the hook.register function, this function "load_hook" is called -- ever time a module is loaded with the file name and the module name. local s_msgA = {} local function isnotempty(s) return s ~= nil and s ~= '' end function load_hook(t) local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn}) ... end Best regards / met vriendelijke groeten Thomas Eylenbosch DevOps Engineer, Gluo N.V. Email: tho...@pa...<mailto:tho...@pa...> Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium BASF Belgium Coordination Center CommV Scheldelaan 600, 2040 Antwerpen, België RPR Antwerpen (afd. Antwerpen) BTW BE0862.390.376 www.basf.be<http://www.basf.be/> Deutsche Bank AG IBAN: BE43 8262 8044 4801 BIC: DEUTBEBE Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html |
|
From: Robert M. <mc...@ta...> - 2024-10-13 01:02:24
|
If you wish to debug your current code, please read the following new section on how to debug Lmod: https://lmod.readthedocs.io/en/latest/165_debugging_lmod.html However, the internals of Lmod can change between versions so relying on internal can lead to problems. In fact, the internals of how the Lmod determines if a module is visible has changed considerably in 8.7.51+ I believe that I have a better way to know if a module is visible or not. As the updated documentation shows: https://lmod.readthedocs.io/en/latest/300_tracking_module_usage.html local hook = require("Hook") local uname = require("posix").uname local cosmic = require("Cosmic"):singleton() local syshost = cosmic:value("LMOD_SYSHOST") local s_msgT = {} local function load_hook(t) -- the arg t is a table: -- t.modFullName: the module full name: (i.e: gcc/4.7.2) -- t.fn: The file name: (i.e /apps/modulefiles/Core/gcc/4.7.2.lua) -- t.mname: The Module Name object. local isVisible = t.mname:isVisible() I have created and uploaded a new version of Lmod 8.7.53 which includes the new member function MName:isVisible(). If you have a chance, please test Lmod 8.7.53 to see if works for you. Best ________________________________ From: Thomas Eylenbosch via Lmod-users <lmo...@li...> Sent: Friday, October 11, 2024 4:36 AM To: lmo...@li... <lmo...@li...> Subject: [Lmod-users] Lmod mrc function not working anymore Hi Lmod community We have a customized SitePackage.lua in our platform to track the lmod usage of the users. Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system: local mrc = MRC:singleton() local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn}) What has changed in the Lmod package that now did break this functionality? I already tried to search related changes in the lmod source code, but I can’t figure it out. How the SitePackage.lua file looks like: require("strict") require("lmod_system_execute") require("string_utils") local MRC = require("MRC") local mrc = MRC:singleton() local hook = require("Hook") local uname = require("posix").uname local cosmic = require("Cosmic"):singleton() local syshost = cosmic:value("LMOD_SYSHOST") local FrameStk = require("FrameStk") -- By using the hook.register function, this function "load_hook" is called -- ever time a module is loaded with the file name and the module name. local s_msgA = {} local function isnotempty(s) return s ~= nil and s ~= '' end function load_hook(t) local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn}) ... end Best regards / met vriendelijke groeten Thomas Eylenbosch DevOps Engineer, Gluo N.V. Email: tho...@pa...<mailto:tho...@pa...> Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium BASF Belgium Coordination Center CommV Scheldelaan 600, 2040 Antwerpen, België RPR Antwerpen (afd. Antwerpen) BTW BE0862.390.376 www.basf.be<http://www.basf.be/> Deutsche Bank AG IBAN: BE43 8262 8044 4801 BIC: DEUTBEBE Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html |
|
From: Thomas E. <tho...@pa...> - 2024-10-11 12:08:58
|
Hi Lmod community
We have a customized SitePackage.lua in our platform to track the lmod usage of the users.
Since the upgrade of Lmod from 8.7.32 to 8.7.48, the following functions are not working anymore to include the visible modules only in our tracking system:
local mrc = MRC:singleton()
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
What has changed in the Lmod package that now did break this functionality?
I already tried to search related changes in the lmod source code, but I can't figure it out.
How the SitePackage.lua file looks like:
require("strict")
require("lmod_system_execute")
require("string_utils")
local MRC = require("MRC")
local mrc = MRC:singleton()
local hook = require("Hook")
local uname = require("posix").uname
local cosmic = require("Cosmic"):singleton()
local syshost = cosmic:value("LMOD_SYSHOST")
local FrameStk = require("FrameStk")
-- By using the hook.register function, this function "load_hook" is called
-- ever time a module is loaded with the file name and the module name.
local s_msgA = {}
local function isnotempty(s)
return s ~= nil and s ~= ''
end
function load_hook(t)
local isVisible = mrc:isVisible({fullName=t.modFullName, sn=t.sn, fn=t.fn})
...
end
Best regards / met vriendelijke groeten
Thomas Eylenbosch
DevOps Engineer, Gluo N.V.
Email: tho...@pa...<mailto:tho...@pa...>
Postal Address: BASF Belgium Coordination Center CommV, Technologiepark 101, 9052 Gent Zwijnaarde, Belgium
BASF Belgium Coordination Center CommV
Scheldelaan 600, 2040 Antwerpen, België
RPR Antwerpen (afd. Antwerpen)
BTW BE0862.390.376
www.basf.be<http://www.basf.be>
Deutsche Bank AG
IBAN: BE43 8262 8044 4801
BIC: DEUTBEBE
Information on data protection can be found here: https://www.basf.com/global/en/legal/data-protection-at-basf.html
|
|
From: Robert M. <mc...@ta...> - 2024-10-09 22:18:22
|
This release of Lmod 8.7.51. The big part of this release is the addition of hide{} and forbid{} function that support hiding and forbidding modules. If you are interested in these features, please test this version out. This change will be released as part of Lmod 8.8 as part of the SC 24 presentation on Nov 19,20. 2024
There are many updates as part of this release:
* Report dofile() usage as an error.
* Updated FPATH support: bash, ksh just add path (init/ksh_funcs) to FPATH
zsh: if autoload && compinit fail then set __zsh_fpath with sub-shell
* improve addto to not include duplicates in path like variables (PATH, FPATH. ,,. )
* Add support for --dumpname in lmod and ml; Update tab completions files
* PR #727: Do not reset BASH_ENV if already set for cshrc.in
* Do not reset BASH_ENV if already set for profile.in
*
Support for hide{} and forbid{}
See https://lmod.readthedocs.io/en/latest/093_modulerc.html#hide-a-more-powerful-way-to-hide-modules
for more details about the hide{} and forbid{} functions.
Best,
Lmod Team.
|
|
From: Robert M. <mc...@ta...> - 2024-10-07 19:17:24
|
Lmod Version 8.7.33 changed it so that warning do not produce an error status.
Best,
Robert
________________________________
From: Rahaman, Ronald O <rra...@ga...>
Sent: Monday, October 7, 2024 9:48 AM
To: Robert McLay <mc...@ta...>; lmod-users <lmo...@li...>
Subject: Re: Configure lmod to return 0 on warnings?
Thanks, I will see if an update to 8.7.49 resolves the issue. We’re still on 8.7.32, and here is what we see:
$ module --version
Modules based on Lua: Version 8.7.32 2023-08-28 12:42 -05:00
by Robert McLay mc...@ta...<mailto:mc...@ta...>
$ module whatis foo
Lmod Warning: Failed to find the following module(s): "foo" in your MODULEPATH
Try:
$ module spider foo
to see if the module(s) are available across all compilers and MPI implementations.
pace-rrahaman6@atl1-1-03-002-35-0:~$ echo $?
1
Best,
Ron
From: Robert McLay <mc...@ta...>
Date: Friday, October 4, 2024 at 9:25 PM
To: Rahaman, Ronald O <rra...@ga...>, lmod-users <lmo...@li...>
Subject: Re: Configure lmod to return 0 on warnings?
I am unclear on what you are asking. As far as I know, the latest version of Lmod, the command:
% module load foo
generates an error and sets the status to 1. This is for a module that does not exist.
On the other hand:
% module whatis foo
Generates a warning and returns 0 as the exit status for a module that does not exist.
The latest version of Lmod has warnings set the return status as 0 and the error exit status as 1. The latest version of Lmod is 8.7.49. Please report issues with respect to that version.
Do you have an example of an Lmod warning that produces an exit status that is non-zero?
Best,
Robert.
________________________________
From: Rahaman, Ronald O <rra...@ga...>
Sent: Friday, October 4, 2024 11:54 AM
To: lmo...@li... <lmo...@li...>
Subject: [Lmod-users] Configure lmod to return 0 on warnings?
Hi all,
Is there a way to configure lmod such that warnings from “module load” and other commands return 0 instead of 1? That would simplify some of our regression tests, which currently fail on lmod warnings that aren’t actually fatal.
Thanks,
Ron
--------
Ron Rahaman
Research Scientist II, Research Software Engineer
Partnership for an Advanced Computing Environment (PACE)
Open Source Programming Office (OSPO)
Georgia Institute of Technology
|
|
From: Rahaman, R. O <rra...@ga...> - 2024-10-07 14:48:32
|
Thanks, I will see if an update to 8.7.49 resolves the issue. We’re still on 8.7.32, and here is what we see:
$ module --version
Modules based on Lua: Version 8.7.32 2023-08-28 12:42 -05:00
by Robert McLay mc...@ta...<mailto:mc...@ta...>
$ module whatis foo
Lmod Warning: Failed to find the following module(s): "foo" in your MODULEPATH
Try:
$ module spider foo
to see if the module(s) are available across all compilers and MPI implementations.
pace-rrahaman6@atl1-1-03-002-35-0:~$ echo $?
1
Best,
Ron
From: Robert McLay <mc...@ta...>
Date: Friday, October 4, 2024 at 9:25 PM
To: Rahaman, Ronald O <rra...@ga...>, lmod-users <lmo...@li...>
Subject: Re: Configure lmod to return 0 on warnings?
I am unclear on what you are asking. As far as I know, the latest version of Lmod, the command:
% module load foo
generates an error and sets the status to 1. This is for a module that does not exist.
On the other hand:
% module whatis foo
Generates a warning and returns 0 as the exit status for a module that does not exist.
The latest version of Lmod has warnings set the return status as 0 and the error exit status as 1. The latest version of Lmod is 8.7.49. Please report issues with respect to that version.
Do you have an example of an Lmod warning that produces an exit status that is non-zero?
Best,
Robert.
________________________________
From: Rahaman, Ronald O <rra...@ga...>
Sent: Friday, October 4, 2024 11:54 AM
To: lmo...@li... <lmo...@li...>
Subject: [Lmod-users] Configure lmod to return 0 on warnings?
Hi all,
Is there a way to configure lmod such that warnings from “module load” and other commands return 0 instead of 1? That would simplify some of our regression tests, which currently fail on lmod warnings that aren’t actually fatal.
Thanks,
Ron
--------
Ron Rahaman
Research Scientist II, Research Software Engineer
Partnership for an Advanced Computing Environment (PACE)
Open Source Programming Office (OSPO)
Georgia Institute of Technology
|
|
From: Robert M. <mc...@ta...> - 2024-10-05 02:25:16
|
I am unclear on what you are asking. As far as I know, the latest version of Lmod, the command: % module load foo generates an error and sets the status to 1. This is for a module that does not exist. On the other hand: % module whatis foo Generates a warning and returns 0 as the exit status for a module that does not exist. The latest version of Lmod has warnings set the return status as 0 and the error exit status as 1. The latest version of Lmod is 8.7.49. Please report issues with respect to that version. Do you have an example of an Lmod warning that produces an exit status that is non-zero? Best, Robert. ________________________________ From: Rahaman, Ronald O <rra...@ga...> Sent: Friday, October 4, 2024 11:54 AM To: lmo...@li... <lmo...@li...> Subject: [Lmod-users] Configure lmod to return 0 on warnings? Hi all, Is there a way to configure lmod such that warnings from “module load” and other commands return 0 instead of 1? That would simplify some of our regression tests, which currently fail on lmod warnings that aren’t actually fatal. Thanks, Ron -------- Ron Rahaman Research Scientist II, Research Software Engineer Partnership for an Advanced Computing Environment (PACE) Open Source Programming Office (OSPO) Georgia Institute of Technology |
|
From: Rahaman, R. O <rra...@ga...> - 2024-10-05 01:33:34
|
Hi all, Is there a way to configure lmod such that warnings from “module load” and other commands return 0 instead of 1? That would simplify some of our regression tests, which currently fail on lmod warnings that aren’t actually fatal. Thanks, Ron -------- Ron Rahaman Research Scientist II, Research Software Engineer Partnership for an Advanced Computing Environment (PACE) Open Source Programming Office (OSPO) Georgia Institute of Technology |
|
From: Thompson, M. (GSFC-610.1)[S. S. A. A. INC]
<mat...@na...> - 2024-10-03 20:01:23
|
Lev, Good idea. I'll ask if that is a viable solution. I used to do that on my personal laptop but then, well, got use to typing out the full `ml` line. Psychology is odd, not sure what changed! Matt -- Matt Thompson, SSAI, Sr Scientific Programmer/Analyst NASA GSFC, Global Modeling and Assimilation Office Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 Phone: 301-614-6712 Fax: 301-614-6246 http://science.gsfc.nasa.gov/sed/bio/matthew.thompson From: Lev Gorenstein <le...@gl...> Date: Thursday, October 3, 2024 at 1:52 PM To: Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] <mat...@na...> Cc: lmod-users <lmo...@li...> Subject: [EXTERNAL] Re: [Lmod-users] Help with "aliasing" MPI (not sure the right word) CAUTION: This email originated from outside of NASA. Please take care when clicking links or opening attachments. Use the "Report Message" button to report suspicious messages to the NASA SOC. Matt, What about just making a collection of Core "meta-modules" called '{gcc,intel,nvhpc}-stack', and have each of them load their corresponding explicit compiler+MPI combinations? The names would be easily recognized and associated with the underlying stack. And to the point of your annoyance with symlinks, in this scenario the 'show' or 'list' commands would show exactly what has been loaded. Lev On Thu, Oct 3, 2024 at 12:38 PM Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC] via Lmod-users <lmo...@li...<mailto:lmo...@li...>> wrote: All, I've had a request on a system I maintain the lmods for and I'm wondering if there is a clever/elegant way to do what they want. To wit, on my system I have, say a variety of compilers and MPI stacks built by those compilers. Core/ ├── gcc │ ├── 14.2.0.lua ├── ifort │ ├── 2021.13.0.lua ├── ifx │ └── 2024.2.lua ├── nag │ └── 7.2.13.lua ├── nvhpc │ └── 24.9.lua Compiler/ ├── gcc-14.2.0 │ └── openmpi │ └── 5.0.4.lua ├── ifort-2021.13.0 │ └── intelmpi │ └── 2021.13.lua ├── ifx-2024.2 │ └── intelmpi │ └── 2021.13.lua │ └── 5.0.4.lua ├── nag-7.2.13 │ ├── mpich │ │ └── 4.2.2.lua │ └── openmpi │ └── 5.0.3-GillesNAGPatch.lua └── nvhpc-24.9 └── openmpi └── 4.1.6.lua What the user would like is to not have to know that, say, Intel means: ml ifort intelmpi but for gfortran it is: ml gcc openmpi They want to do: ml intel mpi ml gcc mpi so they don't need to have that knowledge. And I admit, it isn't really *necessary* for most users to know what MPI stack they have, just that they have one that works. Now, the simple thing would be to just symlink say an openmpi directory to mpi: └── nvhpc-24.9 ├── mpi -> openmpi └── openmpi └── 4.1.6.lua and that does work: > ml nvhpc mpi > ml -t nvhpc/24.9 mpi/4.1.6 but it also sort of...annoys me that it says "mpi/4.1.6" since I like to know what MPI stack I might be using (and it helps supporting users). So I was wondering if there is some more clever way to do this? I mean, I can always just ignore the symlinks myself (and I'd know what stack it is just by the version number), but just in case I'm missing some cool lmod feature/trick, I thought I'd ask. Thanks, Matt -- Matt Thompson, SSAI, Sr Scientific Programmer/Analyst NASA GSFC, Global Modeling and Assimilation Office Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771 Phone: 301-614-6712 Fax: 301-614-6246 http://science.gsfc.nasa.gov/sed/bio/matthew.thompson _______________________________________________ Lmod-users mailing list Lmo...@li...<mailto:Lmo...@li...> https://lists.sourceforge.net/lists/listinfo/lmod-users -- Lev Gorenstein Solutions Architect Globus // University of Chicago e: le...@gl...<mailto:le...@gl...> |
|
From: Lev G. <le...@gl...> - 2024-10-03 19:26:55
|
Matt,
What about just making a collection of Core "meta-modules" called
*'{gcc,intel,nvhpc}-stack'*, and have each of them load their corresponding
explicit compiler+MPI combinations?
The names would be easily recognized and associated with the underlying
stack. And to the point of your annoyance with symlinks, in this scenario
the 'show' or 'list' commands would show exactly what has been loaded.
Lev
On Thu, Oct 3, 2024 at 12:38 PM Thompson, Matt (GSFC-610.1)[SCIENCE SYSTEMS
AND APPLICATIONS INC] via Lmod-users <lmo...@li...>
wrote:
> All,
>
>
>
> I've had a request on a system I maintain the lmods for and I'm wondering
> if there is a clever/elegant way to do what they want.
>
>
>
> To wit, on my system I have, say a variety of compilers and MPI stacks
> built by those compilers.
>
>
>
> Core/
>
> ├── gcc
>
> │ ├── 14.2.0.lua
>
> ├── ifort
>
> │ ├── 2021.13.0.lua
>
> ├── ifx
>
> │ └── 2024.2.lua
>
> ├── nag
>
> │ └── 7.2.13.lua
>
> ├── nvhpc
>
> │ └── 24.9.lua
>
>
>
> Compiler/
>
> ├── gcc-14.2.0
>
> │ └── openmpi
>
> │ └── 5.0.4.lua
>
> ├── ifort-2021.13.0
>
> │ └── intelmpi
>
> │ └── 2021.13.lua
>
> ├── ifx-2024.2
>
> │ └── intelmpi
>
> │ └── 2021.13.lua
>
> │ └── 5.0.4.lua
>
> ├── nag-7.2.13
>
> │ ├── mpich
>
> │ │ └── 4.2.2.lua
>
> │ └── openmpi
>
> │ └── 5.0.3-GillesNAGPatch.lua
>
> └── nvhpc-24.9
>
> └── openmpi
>
> └── 4.1.6.lua
>
>
>
> What the user would like is to not have to know that, say, Intel means:
>
>
>
> ml ifort intelmpi
>
>
>
> but for gfortran it is:
>
>
>
> ml gcc openmpi
>
>
>
> They want to do:
>
>
>
> ml intel mpi
>
> ml gcc mpi
>
>
>
> so they don't need to have that knowledge. And I admit, it isn't really *
> *necessary** for most users to know what MPI stack they have, just that
> they have one that works.
>
>
>
> Now, the simple thing would be to just symlink say an openmpi directory to
> mpi:
>
>
>
> └── nvhpc-24.9
>
> ├── mpi -> openmpi
>
> └── openmpi
>
> └── 4.1.6.lua
>
>
>
> and that does work:
>
>
>
> > ml nvhpc mpi
>
> > ml -t
>
> nvhpc/24.9
>
> mpi/4.1.6
>
>
>
> but it also sort of...annoys me that it says "mpi/4.1.6" since I like to
> know what MPI stack I might be using (and it helps supporting users).
>
>
>
> So I was wondering if there is some more clever way to do this? I mean, I
> can always just ignore the symlinks myself (and I'd know what stack it is
> just by the version number), but just in case I'm missing some cool lmod
> feature/trick, I thought I'd ask.
>
>
>
> Thanks,
>
> Matt
>
> --
>
> Matt Thompson, SSAI, Sr Scientific Programmer/Analyst
>
> NASA GSFC, Global Modeling and Assimilation Office
>
> Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
>
> Phone: 301-614-6712 Fax: 301-614-6246
>
> http://science.gsfc.nasa.gov/sed/bio/matthew.thompson
> _______________________________________________
> Lmod-users mailing list
> Lmo...@li...
> https://lists.sourceforge.net/lists/listinfo/lmod-users
>
--
Lev Gorenstein
Solutions Architect
Globus // University of Chicago
e: le...@gl...
|
|
From: Thompson, M. (GSFC-610.1)[S. S. A. A. INC]
<mat...@na...> - 2024-10-03 16:38:01
|
All,
I've had a request on a system I maintain the lmods for and I'm wondering if there is a clever/elegant way to do what they want.
To wit, on my system I have, say a variety of compilers and MPI stacks built by those compilers.
Core/
├── gcc
│ ├── 14.2.0.lua
├── ifort
│ ├── 2021.13.0.lua
├── ifx
│ └── 2024.2.lua
├── nag
│ └── 7.2.13.lua
├── nvhpc
│ └── 24.9.lua
Compiler/
├── gcc-14.2.0
│ └── openmpi
│ └── 5.0.4.lua
├── ifort-2021.13.0
│ └── intelmpi
│ └── 2021.13.lua
├── ifx-2024.2
│ └── intelmpi
│ └── 2021.13.lua
│ └── 5.0.4.lua
├── nag-7.2.13
│ ├── mpich
│ │ └── 4.2.2.lua
│ └── openmpi
│ └── 5.0.3-GillesNAGPatch.lua
└── nvhpc-24.9
└── openmpi
└── 4.1.6.lua
What the user would like is to not have to know that, say, Intel means:
ml ifort intelmpi
but for gfortran it is:
ml gcc openmpi
They want to do:
ml intel mpi
ml gcc mpi
so they don't need to have that knowledge. And I admit, it isn't really *necessary* for most users to know what MPI stack they have, just that they have one that works.
Now, the simple thing would be to just symlink say an openmpi directory to mpi:
└── nvhpc-24.9
├── mpi -> openmpi
└── openmpi
└── 4.1.6.lua
and that does work:
> ml nvhpc mpi
> ml -t
nvhpc/24.9
mpi/4.1.6
but it also sort of...annoys me that it says "mpi/4.1.6" since I like to know what MPI stack I might be using (and it helps supporting users).
So I was wondering if there is some more clever way to do this? I mean, I can always just ignore the symlinks myself (and I'd know what stack it is just by the version number), but just in case I'm missing some cool lmod feature/trick, I thought I'd ask.
Thanks,
Matt
--
Matt Thompson, SSAI, Sr Scientific Programmer/Analyst
NASA GSFC, Global Modeling and Assimilation Office
Code 610.1, 8800 Greenbelt Rd, Greenbelt, MD 20771
Phone: 301-614-6712 Fax: 301-614-6246
http://science.gsfc.nasa.gov/sed/bio/matthew.thompson
|
|
From: Robert M. <mc...@ta...> - 2024-09-30 19:04:30
|
We will host our next Lmod user meeting on Tuesday, August 6th, at 9:30 CDT (14:30 UTC). Current Agenda: - Q/A - Continued discussion of hide/forbid functions - Support for setenv(), prepend_path like functions that do not reverse on unload? - Support for "module --dumpname" ? - Discussion of Lmod issues over the past month The Zoom meeting link is https://utexas.zoom.us/j/2714596735 You do not need to be an Lmod expert to attend. Best, Robert |
20 messages has been excluded from this view by a project administrator.
