|
From: Robert M. <mc...@ta...> - 2023-11-27 20:17:55
|
What Ward says is all true. If you want to make "module avail" faster, you'll need to use the spider cache. The reason is that Lmod looks for properties inside EACH module. This will slow things down. If you want a fair comparison on performance, you can run $ module -t avail on both Tmod and Lmod. Using the terse option will have both programs do the same amount of work. Best, Robert ________________________________ From: Ward Poelmans <wpo...@gm...> Sent: Monday, November 27, 2023 2:01 PM To: lmo...@li... <lmo...@li...> Subject: Re: [Lmod-users] Lmod is considerably slower than the TCL-based environment modules implementation when running 'module avail' Hi, It's honestly very hard to make any meaningful comment on this. Lmod should always be slightly slower with Tcl modules compared to native lua modules as it needs to convert Tcl to lua. There is scripted and binary version of the converter. The latter should be faster. See https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flmod.readthedocs.io%2Fen%2Flatest%2F095_tcl2lua.html&data=05%7C01%7C%7C6c43b7fcc9744e82931a08dbef83d3b4%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C638367121726302953%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=vzPuqS7PzvlZ8ZjnRkp%2B5T%2BqX5QwrjvcjFIRc2MRGvU%3D&reserved=0<https://lmod.readthedocs.io/en/latest/095_tcl2lua.html> Using a spider cache (and cached loads) will also help: https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flmod.readthedocs.io%2Fen%2Flatest%2F320_improving_perf.html&data=05%7C01%7C%7C6c43b7fcc9744e82931a08dbef83d3b4%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C638367121726302953%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=e66a%2BMLxOnD6CEB2G1ywHMl9OjbN2lCtf4m4V3ZlgSQ%3D&reserved=0<https://lmod.readthedocs.io/en/latest/320_improving_perf.html> But we have no clue what you're exactly testing and in absolute numbers the performance of Lmod is still OK. Ward On 22/11/2023 16:04, 王昊然 wrote: > Hello. > > I'm currently using the TCL-based environment modules implementation, and want > try out Lmod with my existing modulefiles. > > I built the latest release of Lmod and wrote a modulefile for it, so I can > switch the implementation to Lmod via a simple 'module load lmod' command. > This works fine and Lmod supported most TCL modulefiles without problem; but > then I noticed it is much slower when I trying to TAB-complete the module name > in command 'module load ' in bash(1). By using 'set -x' I learned that both > implementation used 'avail' subcommand to retrieve available module names for > completion. > > Here's the speed comparison of 'module avail': > > > $ module --version > Modules Release 5.3.1 (2023-06-27) > $ time module avail > /dev/null 2>&1 > > real 0m0.194s > user 0m0.066s > sys 0m0.022s > $ time module avail > /dev/null 2>&1 > > real 0m0.071s > user 0m0.054s > sys 0m0.007s > $ module load lmod > $ module --version > > Modules based on Lua: Version 8.7 2022-04-16 13:49 -05:00 > by Robert McLay mc...@ta... > > $ time module avail > /dev/null 2>&1 > > real 0m0.630s > user 0m0.561s > sys 0m0.049s > $ time module avail > /dev/null 2>&1 > > real 0m0.613s > user 0m0.550s > sys 0m0.042s > $ time module avail > /dev/null 2>&1 > > real 0m0.623s > user 0m0.562s > sys 0m0.037s > > > _______________________________________________ > Lmod-users mailing list > Lmo...@li... > https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Flmod-users&data=05%7C01%7C%7C6c43b7fcc9744e82931a08dbef83d3b4%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C638367121726302953%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zh7Zmb0f8SMwVYX6Uq4E5uttQcwQuDIjUmxYoWOy%2F9I%3D&reserved=0<https://lists.sourceforge.net/lists/listinfo/lmod-users> _______________________________________________ Lmod-users mailing list Lmo...@li... https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Flmod-users&data=05%7C01%7C%7C6c43b7fcc9744e82931a08dbef83d3b4%7C31d7e2a5bdd8414e9e97bea998ebdfe1%7C0%7C0%7C638367121726302953%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zh7Zmb0f8SMwVYX6Uq4E5uttQcwQuDIjUmxYoWOy%2F9I%3D&reserved=0<https://lists.sourceforge.net/lists/listinfo/lmod-users> >> This message is from an external sender. Learn more about why this << >> matters at https://links.utexas.edu/rtyclf. << |
