Re: [Lmod-users] Module path from new install and spider caching
A Lua based environment module system that reads TCL modulefiles.
Brought to you by:
rtmclay
Menu
▾
▴
Re: [Lmod-users] Module path from new install and spider caching
|
From: Bennet F. <be...@um...> - 2016-02-07 17:22:00
|
Robert,
Yes, it definitely helped! It's the bit about the spider being smart
enough to discern that there are 'other trees' that need walking that
I was not getting. We have a similar script, but it is not so smart.
Thanks for the explanation.
-- bennet
On Sun, Feb 7, 2016 at 11:51 AM, Robert McLay <mc...@ta...> wrote:
> If you have a flat module layout then all the paths in MODULEPATH should be
> the argument to update_lmod_system_cache_files.
>
> If you have a hierarchy then only the paths that are "Core" should be the
> argument. The point of the spider cache is to walk the module tree. If you
> have the compilers in the Core part of the tree then spider will find the
> changes in MODULEPATH and find the compiler dependent modules. The same
> will hold true for the mpi dependent modules as well. You should look at
> the spider cache files. If you look at the entry that contains the compiler
> has a children entry which are the compiler dependent modules. Below is a
> small portion on my computer:
>
> ["/opt/apps/modulefiles/Core/gcc/4.8.1.lua"] = {
> ["Category"] = "Compiler",
> ["Description"] = "Gnu Compiler Collection",
> ["Keyword"] = "Compiler",
> ["Name"] = "gcc",
> ["URL"] = "http://gcc.gnu.org",
> children = {
> ["/opt/apps/modulefiles/Compiler/gcc/4.8"] = {
> ["/opt/apps/modulefiles/Compiler/gcc/4.8/boost/1.55.0.lua"] = {
> ["Category"] = "System Environment/Base",
>
>
> I hope this helps
> R.
>
>
>
>
>
> On Sun, Feb 7, 2016 at 10:40 AM, Bennet Fauber <be...@um...> wrote:
>>
>> That seems to work.
>>
>> [bennet@studentpc3 source]$
>> /sw/arc/centos7/lmod/6.0.29/libexec/update_lmod_system_cache_files \
>> -t /sw/arc/lmod/spider_cache/last_spider.txt \
>> -d /sw/arc/lmod/spider_cache
>> ERROR: $BASE_MODULEPATH is undefined
>>
>>
>> [bennet@studentpc3 source]$
>> /sw/arc/centos7/lmod/6.0.29/libexec/update_lmod_system_cache_files \
>> -t /sw/arc/lmod/spider_cache/last_spider.txt \
>> -d /sw/arc/lmod/spider_cache \
>> /sw/arc/centos7/modulefiles
>>
>> as does
>>
>> [bennet@studentpc3 source]$
>> /sw/arc/centos7/lmod/6.0.29/libexec/update_lmod_system_cache_files \
>> -t /sw/arc/lmod/spider_cache/last_spider.txt \
>> -d /sw/arc/lmod/spider_cache \
>> $MODULEPATH
>>
>> I guess I'm still just a bit confused about what is intended to go
>> into the cache? Just those modules that are independent of other
>> things? That's how I'm interpreting the Core directory. But what
>> about things that are under the compilers or MPI? Do people not cache
>> those?
>>
>> We were sort of thinking that the spider would be most useful to show
>> all the software available in all the various hierarchies and
>> families, and that its results would then be used to find which things
>> need to be loaded to get to each package (if any). In our current,
>> completely flat scheme, people can see everything and then work
>> backwards through the error messages generated by prereq to determine
>> all the prerequisites. Maybe I'm overthinking this. Sorry.
>>
>> -- bennet
>>
>>
>>
>> On Sun, Feb 7, 2016 at 11:15 AM, Trey Dockendorf <tre...@ta...>
>> wrote:
>> > I believe update_lmod_system_cache_files reads BASE_MODULEPATH from
>> > command
>> > line arguments. This is the command I use to update my system cache:
>> >
>> > /apps/lmod/lmod/libexec/update_lmod_system_cache_files -t
>> > /apps/moduleData/system.txt -d /apps/moduleData/cacheDir
>> > /apps/modulefiles/Core
>> >
>> > - Trey
>> >
>> > =============================
>> >
>> > Trey Dockendorf
>> > Systems Analyst I
>> > Texas A&M University
>> > Academy for Advanced Telecommunications and Learning Technologies
>> > Phone: (979)458-2396
>> > Email: tre...@ta...
>> > Jabber: tre...@ta...
>> >
>> > On Sun, Feb 7, 2016 at 9:05 AM, Bennet Fauber <be...@um...> wrote:
>> >>
>> >> I'm still trying to spin this up right, and I am tripping over what
>> >> may be some inconsequential details.
>> >>
>> >> I installed the latest lmod using these
>> >>
>> >> $ git clone https://github.com/TACC/Lmod.git lmod-6.0.29
>> >> $ checkout tags/6.0.29 -b 6.0.29
>> >>
>> >> $ ./configure --prefix=/sw/arc/centos7 \
>> >> --with-module-root-path=/sw/arc/centos7/modulefiles \
>> >> --with-spiderCacheDir=/sw/arc/lmod/spider_cache \
>> >> --with-updateSystemFn=/sw/arc/lmod/spider_cache/last_spider.txt
>> >>
>> >> $ make install
>> >>
>> >> $ ln -s /sw/arc/centos7/lmod/lmod/init/profile
>> >> /etc/profile.d/z00_lmod.sh
>> >> $ ln -s /sw/arc/centos7/lmod/lmod/init/cshrc
>> >> /etc/profile.d/z00_lmod.csh
>> >>
>> >> After doing so and logging in MODULEPATH is set to
>> >>
>> >> $ echo $MODULEPATH
>> >>
>> >>
>> >> /sw/arc/centos7/modulefiles/Linux:/sw/arc/centos7/modulefiles/Core:/sw/arc/centos7/lmod/lmod/modulefiles/Core
>> >>
>> >> I presume that those are just provided as 'examples' and we should
>> >> modify the init scripts and such so that our site is laid out as we
>> >> wish?
>> >>
>> >> I am a little confused still by the spider configuration. I used the
>> >> two configure options, and it appears that those were registered in
>> >> that they appear at the end of
>> >> /sw/arc/centos7/lmod/6.0.29/init/lmodrc.lua,
>> >>
>> >> scDescriptT = {
>> >> {
>> >> ["dir"] = "/sw/arc/lmod/spider_cache",
>> >> ["timestamp"] = "/sw/arc/lmod/spider_cache/last_spider.txt",
>> >> },
>> >> }
>> >>
>> >> but I'm not quite clear on creating the system spider cache. The
>> >> script, update_lmod_system_cache_files, says in its help that
>> >>
>> >> -d <cacheDir>
>> >> location of Lmod cache directory (default: determine via 'ml
>> >> --config')
>> >>
>> >> and the ml --config seems to know about the configured cache and
>> >> timestamp
>> >> file,
>> >>
>> >> $ ml --config 2>&1 | grep -B3 spider
>> >>
>> >> Cache Directory Time Stamp File
>> >> --------------- ---------------
>> >> /sw/arc/lmod/spider_cache /sw/arc/lmod/spider_cache/last_spider.txt
>> >>
>> >> but
>> >>
>> >> $ libexec/update_lmod_system_cache_files
>> >> ERROR: $BASE_MODULEPATH is undefined
>> >>
>> >> $ libexec/update_lmod_system_cache_files -d /sw/arc/lmod/spider_cache
>> >> ERROR: $BASE_MODULEPATH is undefined
>> >>
>> >> and it seems not to recognize that variable even when it does exist,
>> >>
>> >> [bennet@studentpc3 6.0.29]$ export
>> >> BASE_MODULEPATH=/sw/arc/centos7/modulefiles
>> >>
>> >> [bennet@studentpc3 6.0.29]$ libexec/update_lmod_system_cache_files
>> >> ERROR: $BASE_MODULEPATH is undefined
>> >>
>> >> [bennet@studentpc3 6.0.29]$ echo $BASE_MODULEPATH
>> >> /sw/arc/centos7/modulefiles
>> >>
>> >> [bennet@studentpc3 6.0.29]$ ls -ld $BASE_MODULEPATH
>> >> drwxr-sr-x. 3 bennet arcswadm 4096 Feb 7 09:41
>> >> /sw/arc/centos7/modulefiles
>> >>
>> >> I'm sure there's something simple that I'm just overlooking, but I'm
>> >> not able to see it. Pointers?
>> >>
>> >> Thanks, -- bennet
>> >>
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> >> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> >> Monitor end-to-end web transactions and take corrective actions now
>> >> Troubleshoot faster and improve end-user experience. Signup Now!
>> >> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>> >> _______________________________________________
>> >> Lmod-users mailing list
>> >> Lmo...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/lmod-users
>> >
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Site24x7 APM Insight: Get Deep Visibility into Application Performance
>> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
>> Monitor end-to-end web transactions and take corrective actions now
>> Troubleshoot faster and improve end-user experience. Signup Now!
>> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
>> _______________________________________________
>> Lmod-users mailing list
>> Lmo...@li...
>> https://lists.sourceforge.net/lists/listinfo/lmod-users
>
>
>
>
> --
> Robert McLay, Ph.D.
> TACC
> Manager, HPC Software Tools
> (512) 232-8104
>
|
