Re: [Lmod-users] Module path from new install and spider caching
A Lua based environment module system that reads TCL modulefiles.
Brought to you by:
rtmclay
Menu
▾
▴
Re: [Lmod-users] Module path from new install and spider caching
|
From: Robert M. <mc...@ta...> - 2016-02-07 16:52:00
|
If you have a flat module layout then all the paths in MODULEPATH should be
the argument to update_lmod_system_cache_files.
If you have a hierarchy then only the paths that are "Core" should be the
argument. The point of the spider cache is to walk the module tree. If
you have the compilers in the Core part of the tree then spider will find
the changes in MODULEPATH and find the compiler dependent modules. The
same will hold true for the mpi dependent modules as well. You should look
at the spider cache files. If you look at the entry that contains the
compiler has a children entry which are the compiler dependent modules.
Below is a small portion on my computer:
["/opt/apps/modulefiles/Core/gcc/4.8.1.lua"] = {
["Category"] = "Compiler",
["Description"] = "Gnu Compiler Collection",
["Keyword"] = "Compiler",
["Name"] = "gcc",
["URL"] = "http://gcc.gnu.org",
children = {
["/opt/apps/modulefiles/Compiler/gcc/4.8"] = {
["/opt/apps/modulefiles/Compiler/gcc/4.8/boost/1.55.0.lua"] = {
["Category"] = "System Environment/Base",
I hope this helps
R.
On Sun, Feb 7, 2016 at 10:40 AM, Bennet Fauber <be...@um...> wrote:
> That seems to work.
>
> [bennet@studentpc3 source]$
> /sw/arc/centos7/lmod/6.0.29/libexec/update_lmod_system_cache_files \
> -t /sw/arc/lmod/spider_cache/last_spider.txt \
> -d /sw/arc/lmod/spider_cache
> ERROR: $BASE_MODULEPATH is undefined
>
>
> [bennet@studentpc3 source]$
> /sw/arc/centos7/lmod/6.0.29/libexec/update_lmod_system_cache_files \
> -t /sw/arc/lmod/spider_cache/last_spider.txt \
> -d /sw/arc/lmod/spider_cache \
> /sw/arc/centos7/modulefiles
>
> as does
>
> [bennet@studentpc3 source]$
> /sw/arc/centos7/lmod/6.0.29/libexec/update_lmod_system_cache_files \
> -t /sw/arc/lmod/spider_cache/last_spider.txt \
> -d /sw/arc/lmod/spider_cache \
> $MODULEPATH
>
> I guess I'm still just a bit confused about what is intended to go
> into the cache? Just those modules that are independent of other
> things? That's how I'm interpreting the Core directory. But what
> about things that are under the compilers or MPI? Do people not cache
> those?
>
> We were sort of thinking that the spider would be most useful to show
> all the software available in all the various hierarchies and
> families, and that its results would then be used to find which things
> need to be loaded to get to each package (if any). In our current,
> completely flat scheme, people can see everything and then work
> backwards through the error messages generated by prereq to determine
> all the prerequisites. Maybe I'm overthinking this. Sorry.
>
> -- bennet
>
>
>
> On Sun, Feb 7, 2016 at 11:15 AM, Trey Dockendorf <tre...@ta...>
> wrote:
> > I believe update_lmod_system_cache_files reads BASE_MODULEPATH from
> command
> > line arguments. This is the command I use to update my system cache:
> >
> > /apps/lmod/lmod/libexec/update_lmod_system_cache_files -t
> > /apps/moduleData/system.txt -d /apps/moduleData/cacheDir
> > /apps/modulefiles/Core
> >
> > - Trey
> >
> > =============================
> >
> > Trey Dockendorf
> > Systems Analyst I
> > Texas A&M University
> > Academy for Advanced Telecommunications and Learning Technologies
> > Phone: (979)458-2396
> > Email: tre...@ta...
> > Jabber: tre...@ta...
> >
> > On Sun, Feb 7, 2016 at 9:05 AM, Bennet Fauber <be...@um...> wrote:
> >>
> >> I'm still trying to spin this up right, and I am tripping over what
> >> may be some inconsequential details.
> >>
> >> I installed the latest lmod using these
> >>
> >> $ git clone https://github.com/TACC/Lmod.git lmod-6.0.29
> >> $ checkout tags/6.0.29 -b 6.0.29
> >>
> >> $ ./configure --prefix=/sw/arc/centos7 \
> >> --with-module-root-path=/sw/arc/centos7/modulefiles \
> >> --with-spiderCacheDir=/sw/arc/lmod/spider_cache \
> >> --with-updateSystemFn=/sw/arc/lmod/spider_cache/last_spider.txt
> >>
> >> $ make install
> >>
> >> $ ln -s /sw/arc/centos7/lmod/lmod/init/profile
> /etc/profile.d/z00_lmod.sh
> >> $ ln -s /sw/arc/centos7/lmod/lmod/init/cshrc /etc/profile.d/z00_lmod.csh
> >>
> >> After doing so and logging in MODULEPATH is set to
> >>
> >> $ echo $MODULEPATH
> >>
> >>
> /sw/arc/centos7/modulefiles/Linux:/sw/arc/centos7/modulefiles/Core:/sw/arc/centos7/lmod/lmod/modulefiles/Core
> >>
> >> I presume that those are just provided as 'examples' and we should
> >> modify the init scripts and such so that our site is laid out as we
> >> wish?
> >>
> >> I am a little confused still by the spider configuration. I used the
> >> two configure options, and it appears that those were registered in
> >> that they appear at the end of
> >> /sw/arc/centos7/lmod/6.0.29/init/lmodrc.lua,
> >>
> >> scDescriptT = {
> >> {
> >> ["dir"] = "/sw/arc/lmod/spider_cache",
> >> ["timestamp"] = "/sw/arc/lmod/spider_cache/last_spider.txt",
> >> },
> >> }
> >>
> >> but I'm not quite clear on creating the system spider cache. The
> >> script, update_lmod_system_cache_files, says in its help that
> >>
> >> -d <cacheDir>
> >> location of Lmod cache directory (default: determine via 'ml
> >> --config')
> >>
> >> and the ml --config seems to know about the configured cache and
> timestamp
> >> file,
> >>
> >> $ ml --config 2>&1 | grep -B3 spider
> >>
> >> Cache Directory Time Stamp File
> >> --------------- ---------------
> >> /sw/arc/lmod/spider_cache /sw/arc/lmod/spider_cache/last_spider.txt
> >>
> >> but
> >>
> >> $ libexec/update_lmod_system_cache_files
> >> ERROR: $BASE_MODULEPATH is undefined
> >>
> >> $ libexec/update_lmod_system_cache_files -d /sw/arc/lmod/spider_cache
> >> ERROR: $BASE_MODULEPATH is undefined
> >>
> >> and it seems not to recognize that variable even when it does exist,
> >>
> >> [bennet@studentpc3 6.0.29]$ export
> >> BASE_MODULEPATH=/sw/arc/centos7/modulefiles
> >>
> >> [bennet@studentpc3 6.0.29]$ libexec/update_lmod_system_cache_files
> >> ERROR: $BASE_MODULEPATH is undefined
> >>
> >> [bennet@studentpc3 6.0.29]$ echo $BASE_MODULEPATH
> >> /sw/arc/centos7/modulefiles
> >>
> >> [bennet@studentpc3 6.0.29]$ ls -ld $BASE_MODULEPATH
> >> drwxr-sr-x. 3 bennet arcswadm 4096 Feb 7 09:41
> >> /sw/arc/centos7/modulefiles
> >>
> >> I'm sure there's something simple that I'm just overlooking, but I'm
> >> not able to see it. Pointers?
> >>
> >> Thanks, -- bennet
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> >> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> >> Monitor end-to-end web transactions and take corrective actions now
> >> Troubleshoot faster and improve end-user experience. Signup Now!
> >> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> >> _______________________________________________
> >> Lmod-users mailing list
> >> Lmo...@li...
> >> https://lists.sourceforge.net/lists/listinfo/lmod-users
> >
> >
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> Lmod-users mailing list
> Lmo...@li...
> https://lists.sourceforge.net/lists/listinfo/lmod-users
>
--
Robert McLay, Ph.D.
TACC
Manager, HPC Software Tools
(512) 232-8104
|
