|
From: Rob D. <Rob...@fr...> - 2014-07-22 10:11:40
|
Yes, this is why I think it should be either all commands using the cache or none. Using your example below "module avail" would tell you about "B/foo/1.2" but "module load" would load "A/foo/1.2" this would confuse any user that did not know about the caching behaviour. Rob On 21/07/14 21:45, Robert McLay wrote: > I pointed out earlier that it might be a good strategy to load from > the cache and if that fails to walk the directory tree. I have > decided that this is an extremely bad idea. First let me state the > obvious rule that the using the cache or walking the directory tree > should pick the same module every time. Suppose you have > MODULEPATH=A:B and you are trying to load "foo/1.2". The cache file > thinks the structure is: > > A/ > foo/1.0 > > B/ > foo/1.2 > > But the filesystem has: > > A/ > foo/1.0 foo/1.2 > B/ > foo/1.2 > > So reading the cache file will get foo/1.2 from B where as walking the > directory tree will load foo/1.2 from A. This is one of many > scenarios that fail the rule. As far as I can see, the module > system has to walk the tree somehow to get the right result. If you > only walked the directory tree when modulefile wasn't found in the > cache you'd miss cases like the one above. > > R. > > > On Mon, Jul 21, 2014 at 4:21 AM, Rob Dooley <Rob...@fr... > <mailto:Rob...@fr...>> wrote: > > On 17/07/14 19:35, Robert McLay wrote: >> I agree that module load is done more frequently that module >> avail. The trouble is that as sys-admins we constantly are >> running into installing and testing modules and not all the staff >> are or should be an expert in how modules work. So it is safer >> to always ignore the cache on loads. There is also an >> expectation of speed. When I do an "module avail", I want that >> to be fast because I'm waiting to module loads after the avail. >> If the loads are a little slower it doesn't seem to be as much a >> problem. >> > I see. > > Most of our users do not use the module commands directly and so > they are probably unaware of the avail command. Instead we have a > module file per project that loads other module files and people > know what project they are working on. We also have lots of batch > jobs so consistent behaviour is important to us. > Overall caching is looking less attractive as a speed up for us :-( > > I have thought that compiling our higher level project modules > down to single flat files would eliminate most file system delays > as we would be loading a single file. This approach would: > > 1. Need a "release" step which updated the compiled files when > any module file was changed. > 2. Need the compiled module to maintain the $LOADEDMODULES,etc. > env vars. So that later module commands behaved correctly. > 3. Be a very specialised version of caching of course... > > Rob > >> However, It might be a better strategy to try to load via the >> cache and if it fails then walk the MODULEPATH tree before >> telling the user that the module doesn't exist. It is just more >> code. It is only software, how hard could it be? (;->) >> >> R. >> >> >> >> On Thu, Jul 17, 2014 at 12:04 PM, Rob Dooley >> <Rob...@fr... <mailto:Rob...@fr...>> wrote: >> >> On 10/07/14 11:02, Eric Deveaud wrote: >> > Robert McLay <mc...@ta... >> <mailto:mc...@ta...>> écrit : >> >> ...snipped... >> >> >> 5. Lmod uses the cache only for avail (and spider) but >> not for loading. >> >> This way new modules can be loaded even if they >> can't be seen avail. >> > this is obvious. the cache will be only used for the >> available command. >> > >> >> Hi, >> >> I am puzzled by this, elsewhere (sorry I can't find that >> email) someone >> pointed out that "module load" is done far more often than >> "module >> avail" so it seems like you get more benefit from using the >> cache for >> load operations. >> >> Additionally if avail is working from one list of modules and >> load is >> working from a different list (effectively) then you might >> not get what >> you were expecting which is probably less of an issue for >> someone who is >> creating module files but for someone who is just using >> modules created >> by other people this could come as a bit of a shock? For this >> reason I >> think all commands using cache or nothing using cache is better. >> >> Rob >> >> >> 6. Lmod assumes that system modulefiles are publicly >> readable. >> > I agree. >> > >> > Eric >> > >> > >> ------------------------------------------------------------------------------ >> > Open source business process management suite built on Java >> and Eclipse >> > Turn processes into business applications with Bonita BPM >> Community Edition >> > Quickly connect people, data, and systems into organized >> workflows >> > Winner of BOSSIE, CODIE, OW2 and Gartner awards >> > http://p.sf.net/sfu/Bonitasoft >> > _______________________________________________ >> > Modules-interest mailing list >> > Mod...@li... >> <mailto:Mod...@li...> >> > https://lists.sourceforge.net/lists/listinfo/modules-interest >> >> >> ------------------------------------------------------------------------------ >> Want fast and easy access to all the code in your enterprise? >> Index and >> search up to 200,000 lines of code with a free copy of Black Duck >> Code Sight - the same software that powers the world's >> largest code >> search on Ohloh, the Black Duck Open Hub! Try it now. >> http://p.sf.net/sfu/bds >> _______________________________________________ >> Modules-interest mailing list >> Mod...@li... >> <mailto:Mod...@li...> >> https://lists.sourceforge.net/lists/listinfo/modules-interest >> >> >> >> >> -- >> Robert McLay, Ph.D. >> TACC >> Manager, HPC Software Tools >> (512) 232-8104 <tel:%28512%29%20232-8104> >> >> >> >> ------------------------------------------------------------------------------ >> Want fast and easy access to all the code in your enterprise? Index and >> search up to 200,000 lines of code with a free copy of Black Duck >> Code Sight - the same software that powers the world's largest code >> search on Ohloh, the Black Duck Open Hub! Try it now. >> http://p.sf.net/sfu/bds >> >> >> _______________________________________________ >> Modules-interest mailing list >> Mod...@li... <mailto:Mod...@li...> >> https://lists.sourceforge.net/lists/listinfo/modules-interest > > > ------------------------------------------------------------------------------ > Want fast and easy access to all the code in your enterprise? > Index and > search up to 200,000 lines of code with a free copy of Black Duck > Code Sight - the same software that powers the world's largest code > search on Ohloh, the Black Duck Open Hub! Try it now. > http://p.sf.net/sfu/bds > _______________________________________________ > Modules-interest mailing list > Mod...@li... > <mailto:Mod...@li...> > https://lists.sourceforge.net/lists/listinfo/modules-interest > > > > > -- > Robert McLay, Ph.D. > TACC > Manager, HPC Software Tools > (512) 232-8104 > > > > ------------------------------------------------------------------------------ > Want fast and easy access to all the code in your enterprise? Index and > search up to 200,000 lines of code with a free copy of Black Duck > Code Sight - the same software that powers the world's largest code > search on Ohloh, the Black Duck Open Hub! Try it now. > http://p.sf.net/sfu/bds > > > _______________________________________________ > Modules-interest mailing list > Mod...@li... > https://lists.sourceforge.net/lists/listinfo/modules-interest |
