Re: [Lmod-users] [EXTERNAL] Re: changes to tcl 'break' statement in lmod 8 series
A Lua based environment module system that reads TCL modulefiles.
Brought to you by:
rtmclay
Menu
▾
▴
Re: [Lmod-users] [EXTERNAL] Re: changes to tcl 'break' statement in lmod 8 series
|
From: Robert M. <mc...@ta...> - 2022-04-14 20:17:33
|
Well, I was able to track down how break works in TCL. A break inside a
loop exits the loop. A break outside the loop causes an error. This error
can be caught and cancel the evaluation of the current module. I have run
TCL modules with breaks inside a loop as well as outside a loop and it now
works for me. Please test Lmod 8.6.18 to see if it works for you.
Best,
Robert.
On Thu, Apr 14, 2022 at 10:17 AM Shrader, David Lee <dsh...@la...>
wrote:
> What might be very irritating to deal with would be if a
> break/stop-loading-this-modulefile happens inside a for loop. Maybe a
> different function name would have to be used other that "break"?
>
>
> Thanks again for looking in to it.
>
> David
>
>
> ------------------------------
> *From:* Robert McLay <mc...@ta...>
> *Sent:* Wednesday, April 13, 2022 5:29 PM
> *To:* Shrader, David Lee
> *Cc:* lmod-users
> *Subject:* Re: [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
> statement in lmod 8 series
>
> Break is more tricky then I thought it was. I'll have to figure out how
> break works in TCL and tmod. This is going to take some time. "break" will
> break out of a loop. But if it is a bare "break" it does "I'm not loading
> the current module" thing and I don't know how to do that in TCL yet.
>
> Best,
> Robert.
>
>
> On Wed, Apr 13, 2022 at 5:54 PM Shrader, David Lee <dsh...@la...>
> wrote:
>
>> Hey Robert,
>>
>>
>> I have a follow-up scenario to the issue of "break" in tcl modulefiles
>> with Lmod 8.6+. With the new way of dealing with "break", it is now not
>> possible to break out of a for loop in tcl. Whenever this is used, Lmod
>> just exits, on module load and unload. We've got at least one modulefile
>> that does this at our site, and there may be more (we haven't put the
>> newest Lmod on systems with many more user modulefiles on it).
>>
>>
>> Would a possible solution to this be to just use `--with-fastTCLInterp=no`,
>> as came up earlier in this email chain?
>>
>>
>> Thanks,
>>
>> David
>>
>>
>> ------------------------------
>> *From:* Shrader, David Lee
>> *Sent:* Tuesday, March 22, 2022 4:25 PM
>> *To:* Robert McLay
>> *Cc:* lmod-users
>> *Subject:* Re: [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
>> statement in lmod 8 series
>>
>>
>> That is a good point. Again, thank you for all of the help!
>>
>> David
>>
>>
>> ------------------------------
>> *From:* Robert McLay <mc...@ta...>
>> *Sent:* Tuesday, March 22, 2022 3:43 PM
>> *To:* Shrader, David Lee
>> *Cc:* lmod-users
>> *Subject:* Re: [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
>> statement in lmod 8 series
>>
>> You would still get the "error" message in the case where the module
>> reports the message. It is just that the "error" becomes a warning. This
>> means that the warning is reported and the evaluation of the modulefile
>> continues. The user requested the module be unloaded. The user got what
>> they wanted. This is the best I can do in a bad situation.
>>
>> Best,
>> Robert.
>>
>>
>> On Tue, Mar 22, 2022 at 4:16 PM Shrader, David Lee <dsh...@la...>
>> wrote:
>>
>>> Hey Robert,
>>>
>>>
>>> 8.6.16 does unload that "bad" modulefile, including unsetting the FOO
>>> env variable, and there is no longer the infinite loop when erroneously
>>> loading that "bad" modulefile twice.
>>>
>>>
>>> I could see some use in trying to report an error upon unloading a
>>> modulefile, such as the modulefile no longer having enough information to
>>> do what it might need to. We have one like this where, if another required
>>> modulefile is unloaded before it (this particular setup does not use a
>>> modulefile hierarchy), it doesn't have enough information to construct the
>>> paths it needs to upon unloading, and it errors out because the paths
>>> aren't able to be removed from the required environment variables. An
>>> unload like the one done in 8.6.16 would unload the modulefile without
>>> actually removing the paths. Knowing that the removal failed in some way is
>>> nice to have. Of course, there is another solution to that scenario: store
>>> the needed paths from the load in an env variable and just use those if
>>> they are defined rather than try to reconstruct the path.
>>>
>>>
>>> Thanks,
>>>
>>> David
>>>
>>>
>>> ------------------------------
>>> *From:* Robert McLay <mc...@ta...>
>>> *Sent:* Tuesday, March 22, 2022 1:21 PM
>>> *To:* Shrader, David Lee
>>> *Cc:* lmod-users
>>> *Subject:* Re: [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
>>> statement in lmod 8 series
>>>
>>> Your example "bad" module shows why LmodBreak() in Lua (break in TCL)
>>> makes no sense when unloading. I have changed Lmod to ignore LmodBreak (or
>>> break in TCL) when unloading. Please set Lmod 8.6.16 to see if it works
>>> for you. The documentation has been updated as well.
>>>
>>> Best,
>>> Robert.
>>>
>>>
>>> On Tue, Mar 22, 2022 at 11:40 AM Shrader, David Lee <dsh...@la...>
>>> wrote:
>>>
>>>> Hey Robert,
>>>>
>>>>
>>>> Thank you for the 'module --config' command. I've used 'module
>>>> --config' before; it is good to know that the TCL information is in there
>>>> as well.
>>>>
>>>>
>>>> We went ahead and updated to 8.6.15, just to see how our tcl
>>>> modulefiles behave, and they behave better than in 8.4.10. That is, we do
>>>> not get the "unexpected symbol near '}' error when using a modulefile that
>>>> pairs a 'puts stderr ...' command right before a break command. They also
>>>> seem to properly stop loading at the point 'break' is called.
>>>>
>>>>
>>>> That being said, we did stumble on some strange behavior with a
>>>> semi-bad modulefile (meaning, it shouldn't be used in any serious capacity)
>>>> and a user-side mistaken use of 'module load'. In Lmod 7.8.16, we get
>>>> errors for the aberrant behavior, but in Lmod 8.6.15, we get no errors and,
>>>> in one case, an infinite loop.
>>>>
>>>>
>>>> It is probably easiest to just provide an example. Here is a "bad"
>>>> modulefile ("bad" because it can never be unloaded):
>>>>
>>>>
>>>> $> cat foo3/1.0
>>>> #%Module
>>>>
>>>> catch {set foo $env(FOO)}
>>>> if { [info exists foo] } {
>>>> puts stderr "already set"
>>>> break
>>>> }
>>>> setenv FOO "just me"
>>>>
>>>> If this modulefile is loaded and then unloaded, Lmod 8.6.15 silently
>>>> moves on, and the modulefile is not unloaded:
>>>>
>>>>
>>>> $> module load foo3/1.0
>>>> $> echo $FOO
>>>> just me
>>>> $> module rm foo3/1.0
>>>> already set
>>>> $> echo $FOO
>>>> just me
>>>> $> module list
>>>>
>>>> Currently Loaded Modules:
>>>> 1) foo3/1.0
>>>>
>>>> In Lmod 7.8.16, we get this:
>>>>
>>>>
>>>> $> module load foo3/1.0
>>>> $> echo $FOO
>>>> just me
>>>> $> module rm foo3/1.0
>>>> already set
>>>> Lmod has detected the following error:
>>>> /users/dshrader/modulefiles/common/foo3/1.0: (foo3/1.0):
>>>> While processing the following module(s):
>>>> Module fullname Module Filename
>>>> --------------- ---------------
>>>> foo3/1.0 /users/dshrader/modulefiles/common/foo3/1.0
>>>>
>>>> $> module list
>>>>
>>>> Currently Loaded Modules:
>>>> 1) foo3/1.0
>>>>
>>>> What's more, if one tries to load foo3/1.0 at this point (done
>>>> completely by accident by myself, not through any intelligent choice 😉),
>>>> Lmod 8.6.15 goes in to an infinite loop:
>>>>
>>>>
>>>> $> module list
>>>>
>>>> Currently Loaded Modules:
>>>> 1) foo3/1.0
>>>>
>>>> $> module load foo3/1.0
>>>> already set
>>>> already set
>>>> already set
>>>> already set
>>>> already set
>>>> already set
>>>> ...continues until crtl+c...
>>>>
>>>>
>>>> In Lmod 7.8.16, we get an error:
>>>>
>>>>
>>>> $> module list
>>>>
>>>> Currently Loaded Modules:
>>>> 1) foo3/1.0
>>>>
>>>> $> module load foo3/1.0
>>>> already set
>>>> Lmod has detected the following error:
>>>> /users/dshrader/modulefiles/common/foo3/1.0: (foo3/1.0):
>>>> While processing the following module(s):
>>>> Module fullname Module Filename
>>>> --------------- ---------------
>>>> foo3/1.0 /users/dshrader/modulefiles/common/foo3/1.0
>>>>
>>>>
>>>> So, it is possible to get Lmod 8.6.15 in to an interesting state. For
>>>> myself, I do prefer the error messages from Lmod 7.8.16 over 8.6.15's
>>>> current behavior. Since all of the above is based on a pretty poor
>>>> modulefile, though, I'm not sure how much time should be spent on dealing
>>>> with the above behavior. But, I wanted to at least let you know what we
>>>> found as we were playing around with our tcl modulefiles under 8.6.15. For
>>>> the most part, going to 8.6.15 looks like it will be fairly seamless for
>>>> the existing TCL modulefiles, so we'll push for our OS vendors to update to
>>>> 8.6.15 rather than stay at 8.4.10.
>>>>
>>>>
>>>> Thanks again!
>>>>
>>>> David
>>>>
>>>> ------------------------------
>>>> *From:* Robert McLay <mc...@ta...>
>>>> *Sent:* Monday, March 21, 2022 4:49 PM
>>>> *To:* Shrader, David Lee
>>>> *Cc:* lmod-users
>>>> *Subject:* Re: [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
>>>> statement in lmod 8 series
>>>>
>>>> It is not a bug. There was no support for break in Lmod before 8.6+.
>>>> It is just that having the tcl interpreter as a separate program meant that
>>>> you got away with it.
>>>>
>>>>
>>>> > Do you mean that all of the 8.x versions include their own TCL
>>>> interpreter, and 7.x relies on the system's TCL interpreter?
>>>> Yes
>>>>
>>>> > We typically get Lmod from the OS vendor; is there a quick way to
>>>> find out if the TCL interpreter was included?
>>>>
>>>> $ module --config 2>&1 | grep -i "Use attached TCL"
>>>> Use attached TCL over system call yes
>>>>
>>>> I would be best if you upgraded to Lmod 8.6.15 or later
>>>>
>>>> Best,
>>>> Robert
>>>>
>>>> On Mon, Mar 21, 2022 at 4:06 PM Shrader, David Lee <dsh...@la...>
>>>> wrote:
>>>>
>>>>> Hey Robert,
>>>>>
>>>>>
>>>>> If I understand what you are saying, there should be no difference in
>>>>> how break is handled between 7.x and <8.6? If that is the case, then we may
>>>>> have hit a bug of some sort. But, since the latest version of Lmod is quite
>>>>> a ways beyond what we're looking to get (8.4.10), there may be little
>>>>> utility in trying to track it down. It might just be easier to try to get
>>>>> our upstream to provide an 8.6+...
>>>>>
>>>>>
>>>>> Do you mean that all of the 8.x versions include their own TCL
>>>>> interpreter, and 7.x relies on the system's TCL interpreter?
>>>>>
>>>>>
>>>>> We typically get Lmod from the OS vedor; is there a quick way to find
>>>>> out if the TCL interpreter was included?
>>>>>
>>>>> Thanks again!
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>> ------------------------------
>>>>> *From:* Robert McLay <mc...@ta...>
>>>>> *Sent:* Monday, March 21, 2022 2:40 PM
>>>>> *To:* Shrader, David Lee
>>>>> *Cc:* lmod-users
>>>>> *Subject:* [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
>>>>> statement in lmod 8 series
>>>>>
>>>>> Well yes and no. There is no change in how break is or has been
>>>>> handled since forever. The change in supporting break is in Lmod 8.6+
>>>>>
>>>>> The change is that by default Lmod now includes the TCL interpreter as
>>>>> part of the Lmod executable instead of executing a system call
>>>>> (/usr/bin/tclsh ....). You can configure Lmod to not include the tcl
>>>>> interpreter library as part of Lmod. That should give you the old
>>>>> behavior. Use --with-fastTCLInterp=no to configure Lmod without the tcl
>>>>> integration.
>>>>>
>>>>> I would be interested in seeing how Lmod 8.6+ works with your modules
>>>>> even if you don't deploy it to users.
>>>>>
>>>>>
>>>>> Best,
>>>>> Robert.
>>>>>
>>>>>
>>>>> On Mon, Mar 21, 2022 at 2:20 PM Shrader, David Lee via Lmod-users <
>>>>> lmo...@li...> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>>
>>>>>> We're are potentially going from Lmod 7.8.16 to 8.4.10, and we have
>>>>>> noticed that our tcl modulefiles that call "break" after a "puts stderr
>>>>>> ..." message now generate something like the following error:
>>>>>>
>>>>>>
>>>>>> Lmod has detected the following error: Unable to load module because
>>>>>> of error when evaluating modulefile: foo3
>>>>>> /modulefiles/foo3/1.0: [string "LmodMsgRaw([===[already
>>>>>> set..."]:4: unexpected symbol near '}'
>>>>>>
>>>>>> We have used this method of "puts stderr...; break" to exit out of a
>>>>>> particular modulefile when an error occurs, such as a missing directory.
>>>>>>
>>>>>> In looking over the readthedocs tcl function page, it looks like the
>>>>>> "break" function has seen some attention, but we are looking at 8.4.10, not
>>>>>> 8.6+. Did something change for the "break" command in 8.0 to 8.5 such that
>>>>>> it won't work as we have been using it in 7.x?
>>>>>>
>>>>>>
>>>>>> Thank you very much for any and all help!
>>>>>>
>>>>>> David
>>>>>>
>>>>>>
>>>>>> --
>>>>>> David Shrader
>>>>>> HPC-ENV High Performance Computing Systems
>>>>>> Los Alamos National Lab
>>>>>> _______________________________________________
>>>>>> Lmod-users mailing list
>>>>>> Lmo...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/lmod-users
>>>>>> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Fnam12.safelinks.protection.outlook.com%2F%3Furl%3Dhttps*3A*2F*2Flists.sourceforge.net*2Flists*2Flistinfo*2Flmod-users%26data%3D04*7C01*7C*7C3ba8d73348fc47faf2a708da1da0a814*7C31d7e2a5bdd8414e9e97bea998ebdfe1*7C0*7C0*7C637854873104076991*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000%26sdata%3DJmgo4gnqYaZW4vTr5ocGcBhHKZWEevz4OdFq5xKP1Gw*3D%26reserved%3D0__%3BJSUlJSUlJSUlJSUlJSUlJSUl!!Bt8fGhp8LhKGRg!QOTIYbfOnk97D1Z-OL5Keb3b23zkF2CghPw_3vcxZpR2BHr5MxTmmUyumtmKsTac%24&data=04%7C01%7C%7Cd8836b92dcff4840e79f08da1e29df91%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C637855462444576759%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=iNpfba4CJBaB2kF%2B6vVLzEs3H40st4FCH5bv2nBGIMc%3D&reserved=0>
>>>>>> >> This message is from an external sender. Learn more about why this
>>>>>> <<
>>>>>> >> matters at https://links.utexas.edu/rtyclf
>>>>>> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.com%2Fv3%2F__https%3A%2F%2Flinks.utexas.edu%2Frtyclf__%3B!!Bt8fGhp8LhKGRg!QOTIYbfOnk97D1Z-OL5Keb3b23zkF2CghPw_3vcxZpR2BHr5MxTmmUyumlv7Ge-H%24&data=04%7C01%7C%7Cd8836b92dcff4840e79f08da1e29df91%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C637855462444576759%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Iz9Pqca1VtOdzI8tWCBqs6ImP0gwNNg%2BbIYGNknJ60I%3D&reserved=0>.
>>>>>> <<
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Robert McLay, Ph.D.
>>>>> Manager of Software Tools, HPC
>>>>> mc...@ta...
>>>>>
>>>>>
>>>>
>>>> --
>>>> Robert McLay, Ph.D.
>>>> Manager of Software Tools, HPC
>>>> mc...@ta...
>>>>
>>>>
>>>
>>> --
>>> Robert McLay, Ph.D.
>>> Manager of Software Tools, HPC
>>> mc...@ta...
>>>
>>>
>>
>> --
>> Robert McLay, Ph.D.
>> Manager of Software Tools, HPC
>> mc...@ta...
>>
>>
>
> --
> Robert McLay, Ph.D.
> Manager of Software Tools, HPC
> mc...@ta...
>
>
--
Robert McLay, Ph.D.
Manager of Software Tools, HPC
mc...@ta...
|
