Re: [Lmod-users] [EXTERNAL] Re: changes to tcl 'break' statement in lmod 8 series
A Lua based environment module system that reads TCL modulefiles.
Brought to you by:
rtmclay
Menu
▾
▴
Re: [Lmod-users] [EXTERNAL] Re: changes to tcl 'break' statement in lmod 8 series
|
From: Robert M. <mc...@ta...> - 2022-04-13 23:30:20
|
Break is more tricky then I thought it was. I'll have to figure out how
break works in TCL and tmod. This is going to take some time. "break" will
break out of a loop. But if it is a bare "break" it does "I'm not loading
the current module" thing and I don't know how to do that in TCL yet.
Best,
Robert.
On Wed, Apr 13, 2022 at 5:54 PM Shrader, David Lee <dsh...@la...>
wrote:
> Hey Robert,
>
>
> I have a follow-up scenario to the issue of "break" in tcl modulefiles
> with Lmod 8.6+. With the new way of dealing with "break", it is now not
> possible to break out of a for loop in tcl. Whenever this is used, Lmod
> just exits, on module load and unload. We've got at least one modulefile
> that does this at our site, and there may be more (we haven't put the
> newest Lmod on systems with many more user modulefiles on it).
>
>
> Would a possible solution to this be to just use `--with-fastTCLInterp=no`,
> as came up earlier in this email chain?
>
>
> Thanks,
>
> David
>
>
> ------------------------------
> *From:* Shrader, David Lee
> *Sent:* Tuesday, March 22, 2022 4:25 PM
> *To:* Robert McLay
> *Cc:* lmod-users
> *Subject:* Re: [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
> statement in lmod 8 series
>
>
> That is a good point. Again, thank you for all of the help!
>
> David
>
>
> ------------------------------
> *From:* Robert McLay <mc...@ta...>
> *Sent:* Tuesday, March 22, 2022 3:43 PM
> *To:* Shrader, David Lee
> *Cc:* lmod-users
> *Subject:* Re: [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
> statement in lmod 8 series
>
> You would still get the "error" message in the case where the module
> reports the message. It is just that the "error" becomes a warning. This
> means that the warning is reported and the evaluation of the modulefile
> continues. The user requested the module be unloaded. The user got what
> they wanted. This is the best I can do in a bad situation.
>
> Best,
> Robert.
>
>
> On Tue, Mar 22, 2022 at 4:16 PM Shrader, David Lee <dsh...@la...>
> wrote:
>
>> Hey Robert,
>>
>>
>> 8.6.16 does unload that "bad" modulefile, including unsetting the FOO env
>> variable, and there is no longer the infinite loop when erroneously loading
>> that "bad" modulefile twice.
>>
>>
>> I could see some use in trying to report an error upon unloading a
>> modulefile, such as the modulefile no longer having enough information to
>> do what it might need to. We have one like this where, if another required
>> modulefile is unloaded before it (this particular setup does not use a
>> modulefile hierarchy), it doesn't have enough information to construct the
>> paths it needs to upon unloading, and it errors out because the paths
>> aren't able to be removed from the required environment variables. An
>> unload like the one done in 8.6.16 would unload the modulefile without
>> actually removing the paths. Knowing that the removal failed in some way is
>> nice to have. Of course, there is another solution to that scenario: store
>> the needed paths from the load in an env variable and just use those if
>> they are defined rather than try to reconstruct the path.
>>
>>
>> Thanks,
>>
>> David
>>
>>
>> ------------------------------
>> *From:* Robert McLay <mc...@ta...>
>> *Sent:* Tuesday, March 22, 2022 1:21 PM
>> *To:* Shrader, David Lee
>> *Cc:* lmod-users
>> *Subject:* Re: [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
>> statement in lmod 8 series
>>
>> Your example "bad" module shows why LmodBreak() in Lua (break in TCL)
>> makes no sense when unloading. I have changed Lmod to ignore LmodBreak (or
>> break in TCL) when unloading. Please set Lmod 8.6.16 to see if it works
>> for you. The documentation has been updated as well.
>>
>> Best,
>> Robert.
>>
>>
>> On Tue, Mar 22, 2022 at 11:40 AM Shrader, David Lee <dsh...@la...>
>> wrote:
>>
>>> Hey Robert,
>>>
>>>
>>> Thank you for the 'module --config' command. I've used 'module --config'
>>> before; it is good to know that the TCL information is in there as well.
>>>
>>>
>>> We went ahead and updated to 8.6.15, just to see how our tcl modulefiles
>>> behave, and they behave better than in 8.4.10. That is, we do not get the
>>> "unexpected symbol near '}' error when using a modulefile that pairs a
>>> 'puts stderr ...' command right before a break command. They also seem to
>>> properly stop loading at the point 'break' is called.
>>>
>>>
>>> That being said, we did stumble on some strange behavior with a semi-bad
>>> modulefile (meaning, it shouldn't be used in any serious capacity) and a
>>> user-side mistaken use of 'module load'. In Lmod 7.8.16, we get errors for
>>> the aberrant behavior, but in Lmod 8.6.15, we get no errors and, in one
>>> case, an infinite loop.
>>>
>>>
>>> It is probably easiest to just provide an example. Here is a "bad"
>>> modulefile ("bad" because it can never be unloaded):
>>>
>>>
>>> $> cat foo3/1.0
>>> #%Module
>>>
>>> catch {set foo $env(FOO)}
>>> if { [info exists foo] } {
>>> puts stderr "already set"
>>> break
>>> }
>>> setenv FOO "just me"
>>>
>>> If this modulefile is loaded and then unloaded, Lmod 8.6.15 silently
>>> moves on, and the modulefile is not unloaded:
>>>
>>>
>>> $> module load foo3/1.0
>>> $> echo $FOO
>>> just me
>>> $> module rm foo3/1.0
>>> already set
>>> $> echo $FOO
>>> just me
>>> $> module list
>>>
>>> Currently Loaded Modules:
>>> 1) foo3/1.0
>>>
>>> In Lmod 7.8.16, we get this:
>>>
>>>
>>> $> module load foo3/1.0
>>> $> echo $FOO
>>> just me
>>> $> module rm foo3/1.0
>>> already set
>>> Lmod has detected the following error:
>>> /users/dshrader/modulefiles/common/foo3/1.0: (foo3/1.0):
>>> While processing the following module(s):
>>> Module fullname Module Filename
>>> --------------- ---------------
>>> foo3/1.0 /users/dshrader/modulefiles/common/foo3/1.0
>>>
>>> $> module list
>>>
>>> Currently Loaded Modules:
>>> 1) foo3/1.0
>>>
>>> What's more, if one tries to load foo3/1.0 at this point (done
>>> completely by accident by myself, not through any intelligent choice 😉),
>>> Lmod 8.6.15 goes in to an infinite loop:
>>>
>>>
>>> $> module list
>>>
>>> Currently Loaded Modules:
>>> 1) foo3/1.0
>>>
>>> $> module load foo3/1.0
>>> already set
>>> already set
>>> already set
>>> already set
>>> already set
>>> already set
>>> ...continues until crtl+c...
>>>
>>>
>>> In Lmod 7.8.16, we get an error:
>>>
>>>
>>> $> module list
>>>
>>> Currently Loaded Modules:
>>> 1) foo3/1.0
>>>
>>> $> module load foo3/1.0
>>> already set
>>> Lmod has detected the following error:
>>> /users/dshrader/modulefiles/common/foo3/1.0: (foo3/1.0):
>>> While processing the following module(s):
>>> Module fullname Module Filename
>>> --------------- ---------------
>>> foo3/1.0 /users/dshrader/modulefiles/common/foo3/1.0
>>>
>>>
>>> So, it is possible to get Lmod 8.6.15 in to an interesting state. For
>>> myself, I do prefer the error messages from Lmod 7.8.16 over 8.6.15's
>>> current behavior. Since all of the above is based on a pretty poor
>>> modulefile, though, I'm not sure how much time should be spent on dealing
>>> with the above behavior. But, I wanted to at least let you know what we
>>> found as we were playing around with our tcl modulefiles under 8.6.15. For
>>> the most part, going to 8.6.15 looks like it will be fairly seamless for
>>> the existing TCL modulefiles, so we'll push for our OS vendors to update to
>>> 8.6.15 rather than stay at 8.4.10.
>>>
>>>
>>> Thanks again!
>>>
>>> David
>>>
>>> ------------------------------
>>> *From:* Robert McLay <mc...@ta...>
>>> *Sent:* Monday, March 21, 2022 4:49 PM
>>> *To:* Shrader, David Lee
>>> *Cc:* lmod-users
>>> *Subject:* Re: [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
>>> statement in lmod 8 series
>>>
>>> It is not a bug. There was no support for break in Lmod before 8.6+.
>>> It is just that having the tcl interpreter as a separate program meant that
>>> you got away with it.
>>>
>>>
>>> > Do you mean that all of the 8.x versions include their own TCL
>>> interpreter, and 7.x relies on the system's TCL interpreter?
>>> Yes
>>>
>>> > We typically get Lmod from the OS vendor; is there a quick way to find
>>> out if the TCL interpreter was included?
>>>
>>> $ module --config 2>&1 | grep -i "Use attached TCL"
>>> Use attached TCL over system call yes
>>>
>>> I would be best if you upgraded to Lmod 8.6.15 or later
>>>
>>> Best,
>>> Robert
>>>
>>> On Mon, Mar 21, 2022 at 4:06 PM Shrader, David Lee <dsh...@la...>
>>> wrote:
>>>
>>>> Hey Robert,
>>>>
>>>>
>>>> If I understand what you are saying, there should be no difference in
>>>> how break is handled between 7.x and <8.6? If that is the case, then we may
>>>> have hit a bug of some sort. But, since the latest version of Lmod is quite
>>>> a ways beyond what we're looking to get (8.4.10), there may be little
>>>> utility in trying to track it down. It might just be easier to try to get
>>>> our upstream to provide an 8.6+...
>>>>
>>>>
>>>> Do you mean that all of the 8.x versions include their own TCL
>>>> interpreter, and 7.x relies on the system's TCL interpreter?
>>>>
>>>>
>>>> We typically get Lmod from the OS vedor; is there a quick way to find
>>>> out if the TCL interpreter was included?
>>>>
>>>> Thanks again!
>>>>
>>>> David
>>>>
>>>>
>>>> ------------------------------
>>>> *From:* Robert McLay <mc...@ta...>
>>>> *Sent:* Monday, March 21, 2022 2:40 PM
>>>> *To:* Shrader, David Lee
>>>> *Cc:* lmod-users
>>>> *Subject:* [EXTERNAL] Re: [Lmod-users] changes to tcl 'break'
>>>> statement in lmod 8 series
>>>>
>>>> Well yes and no. There is no change in how break is or has been
>>>> handled since forever. The change in supporting break is in Lmod 8.6+
>>>>
>>>> The change is that by default Lmod now includes the TCL interpreter as
>>>> part of the Lmod executable instead of executing a system call
>>>> (/usr/bin/tclsh ....). You can configure Lmod to not include the tcl
>>>> interpreter library as part of Lmod. That should give you the old
>>>> behavior. Use --with-fastTCLInterp=no to configure Lmod without the tcl
>>>> integration.
>>>>
>>>> I would be interested in seeing how Lmod 8.6+ works with your modules
>>>> even if you don't deploy it to users.
>>>>
>>>>
>>>> Best,
>>>> Robert.
>>>>
>>>>
>>>> On Mon, Mar 21, 2022 at 2:20 PM Shrader, David Lee via Lmod-users <
>>>> lmo...@li...> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>>
>>>>> We're are potentially going from Lmod 7.8.16 to 8.4.10, and we have
>>>>> noticed that our tcl modulefiles that call "break" after a "puts stderr
>>>>> ..." message now generate something like the following error:
>>>>>
>>>>>
>>>>> Lmod has detected the following error: Unable to load module because
>>>>> of error when evaluating modulefile: foo3
>>>>> /modulefiles/foo3/1.0: [string "LmodMsgRaw([===[already
>>>>> set..."]:4: unexpected symbol near '}'
>>>>>
>>>>> We have used this method of "puts stderr...; break" to exit out of a
>>>>> particular modulefile when an error occurs, such as a missing directory.
>>>>>
>>>>> In looking over the readthedocs tcl function page, it looks like the
>>>>> "break" function has seen some attention, but we are looking at 8.4.10, not
>>>>> 8.6+. Did something change for the "break" command in 8.0 to 8.5 such that
>>>>> it won't work as we have been using it in 7.x?
>>>>>
>>>>>
>>>>> Thank you very much for any and all help!
>>>>>
>>>>> David
>>>>>
>>>>>
>>>>> --
>>>>> David Shrader
>>>>> HPC-ENV High Performance Computing Systems
>>>>> Los Alamos National Lab
>>>>> _______________________________________________
>>>>> Lmod-users mailing list
>>>>> Lmo...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/lmod-users
>>>>> <https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Flmod-users&data=04%7C01%7C%7C3ba8d73348fc47faf2a708da1da0a814%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C637854873104076991%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Jmgo4gnqYaZW4vTr5ocGcBhHKZWEevz4OdFq5xKP1Gw%3D&reserved=0>
>>>>> >> This message is from an external sender. Learn more about why this
>>>>> <<
>>>>> >> matters at https://links.utexas.edu/rtyclf.
>>>>> <<
>>>>>
>>>>
>>>>
>>>> --
>>>> Robert McLay, Ph.D.
>>>> Manager of Software Tools, HPC
>>>> mc...@ta...
>>>>
>>>>
>>>
>>> --
>>> Robert McLay, Ph.D.
>>> Manager of Software Tools, HPC
>>> mc...@ta...
>>>
>>>
>>
>> --
>> Robert McLay, Ph.D.
>> Manager of Software Tools, HPC
>> mc...@ta...
>>
>>
>
> --
> Robert McLay, Ph.D.
> Manager of Software Tools, HPC
> mc...@ta...
>
>
--
Robert McLay, Ph.D.
Manager of Software Tools, HPC
mc...@ta...
|
