Menu
▾
▴
Re: [mesa-users] Installing MESA on a supercomputer (Richard Townsend)
|
From: Kevin M. <km...@ph...> - 2012-01-10 19:44:29
|
Success!
I just got MESA compiled and ran through inlist_first_thing_to_try and everything looks to be working fine! It took 59s of cpu time and 13s of wall time to run.
Here's what I had to do to get it working on Triton if anyone else is interested:
Add the following lines to my .bashrc file:
module purge
module load intel
module load openmpi_mx
export MX_RCACHE=0
export LD_LIBRARY_PATH=/opt/intel/Compiler/11.1/072/lib/intel64:$LD_LIBRARY_PATH
Comment out these lines in mesa/adipls/test/ck:
scan-amdl.d amdl.mesa >> .list
check_okay
For some reason, the adipls test was working fine, but there was a weird error when the script called scan-amd.d and this was dumped to the diff.txt file:
forrtl: severe (47): write to READONLY file, unit 5, file stdin
Image PC Routine Line Source
scan-amdl.d.x 000000000047B8ED Unknown Unknown Unknown
scan-amdl.d.x 000000000047A3F5 Unknown Unknown Unknown
scan-amdl.d.x 00000000004348C0 Unknown Unknown Unknown
scan-amdl.d.x 00000000004063BF Unknown Unknown Unknown
scan-amdl.d.x 0000000000405BF2 Unknown Unknown Unknown
scan-amdl.d.x 000000000042E755 Unknown Unknown Unknown
scan-amdl.d.x 0000000000402B6F Unknown Unknown Unknown
scan-amdl.d.x 0000000000402ACC Unknown Unknown Unknown
libc.so.6 00000038F5C1D994 Unknown Unknown Unknown
scan-amdl.d.x 00000000004029D9 Unknown Unknown Unknown
I couldn't track down why this was happening, but commenting out those lines in the test script fixed things.
Then just ./install in the mesa directory as usual (Triton thankfully has ifort 11.1). I haven't tried linking with Triton's MKL libraries yet so I just used the default MESA blas & lapack libraries.
-Kevin
On Jan 7, 2012, at 6:52 PM, Kevin Moore wrote:
> Yes, the purpose of doing this was to do parameter surveys as well as have a centralized location where students in our stars class could run MESA. We get free time on a variety of computing centers so this is a simple solution for us.
>
> Anyway, just an update - I actually got the people at Triton to compile a newer glibc (2.14) for me and now the installation gets all the way to the mtx module (yay?), giving the error:
> /home/kmoore-ucsb/mesa_3794/utils/mesasdk/lib/liblapack.a(xerbla.o): In function `xerbla_':
> xerbla.f:(.text+0x66): undefined reference to `_gfortran_transfer_character_write'
> xerbla.f:(.text+0x76): undefined reference to `_gfortran_transfer_integer_write'
> collect2: ld returned 1 exit status
> make: *** [tester] Error 1
>
> This is enough to convince me I should probably be compiling it from source directly so will try that and post again if it succeeds/fails.
>
> Thanks for all the advice!
>
> -Kevin
>
> On Jan 5, 2012, at 2:51 PM, Richard Townsend wrote:
>
>> Agreed - and in fact I've done these sorts of calculations myself, using Bill's earlier EZ code.
>>
>> But in this context (and based on Ben's remarks about the need for specialized compilers) I was assuming 'supercomputer' means something with specialized hardware, such as a cluster linked together with Infiniband interconnects. While such systems can certainly be used for parameter surveys, a simple Ethernet-networked computer cluster can offer the same performance at a fraction of the price.
>>
>> Cheers,
>>
>> Rich
>>
>> On Jan 5, 2012, at 7:55 AM, Francis Timmes <fx...@ma...> wrote:
>>
>>> hi rich,
>>>
>>> there is lots to be gained by running multiple jobs simultaneously
>>> on a cluster; mesa (or any other non-mpi code) in embarrassingly parallel mode.
>>> for example, a parameter survey involving a 100 stars.
>>>
>>> fxt
>>>
>>>
>>>
>>>
>>> On Jan 5, 2012, at 2:41 AM, Richard Townsend wrote:
>>>
>>>> I'm not even sure what is to be gained from running MESA on a supercomputer. Parallelism in MESA is only via OpenMP - there is no MPIA usage. As such, MESA can take advantage of shared memory architectures, but not clustered architectures.
>>>>
>>>> Cheers,
>>>>
>>>> Rich
>>>>
>>>> On Jan 5, 2012, at 7:31 AM, Francis Timmes <fx...@ma...> wrote:
>>>>
>>>>> hi ben, rich, kevin (and the mesa community),
>>>>>
>>>>> i had no luck compiling mesa under pgi on the nersc
>>>>> machines hopper and franklin this past summer. the issue
>>>>> boiled down to certain pointer structures used in mesa
>>>>> not being supported in that version of the pgi compiler.
>>>>>
>>>>> i would be interested in hearing from anyone about
>>>>> successes or failures on running mesa on any supercomputer
>>>>> or largish parallel cluster.
>>>>>
>>>>> fxt
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Jan 4, 2012, at 3:56 PM, Ben Brown wrote:
>>>>>
>>>>>> Kevin,
>>>>>> To follow up on Rich's note: you are almost certainly not going to be
>>>>>> able to use the MESA SDK on a supercomputer. From my experiences with
>>>>>> those systems (using a very different code), they tend to require highly
>>>>>> specialized libraries and compilers. A general purpose tool like the SDK
>>>>>> won't work, because often the libraries being referenced need to be
>>>>>> compiled against very specific hardware.
>>>>>>
>>>>>> Put differently, the login nodes and the compute nodes tend to run very
>>>>>> different operating systems and codes targeting the compute nodes need to
>>>>>> be compiled using the specific compilers supported by UCSD for Triton.
>>>>>> Which means you'll need to come up with a specialized solution for this
>>>>>> system. I don't have access to Triton myself, so I can't be much more
>>>>>> specific than that. Looking over the Triton webpages it looks like both
>>>>>> PGI and Intel compilers are available on the system:
>>>>>>
>>>>>> http://tritonresource.sdsc.edu/compile_jobs.php
>>>>>>
>>>>>> If you run into problems, you might try contacting Theodore Sande
>>>>>> (formerly at MIT) for help in getting a version of MESA to compile, as he
>>>>>> has had the code running on large-scale systems in the past.
>>>>>>
>>>>>> Otherwise I could take a look at Ranger or Kraken (supercomputers at TACC
>>>>>> and NICS; both have Intel and PGI compilers) later this week or next week.
>>>>>> They won't be a perfect match, but they may get you started.
>>>>>> --Ben
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Hi Kevin --
>>>>>>>
>>>>>>> This is a common problem with systems using 'older' (pre-2.11) versions of the GNU C library (glibc), which is a standard (and significant) part of all Linux distributions. At the moment, I have no plans to support pre-2.11 glibc versions, since 2.11 has been out for over two years. So, the only workaround would be to upgrade to 2.11 or later.
>>>>>>>
>>>>>>> cheers,
>>>>>>>
>>>>>>> Rich
>>>>>>>
>>>>>>> On Jan 4, 2012, at 7:23 PM, Kevin Moore wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I'm trying to install MESA on the Triton computing cluster at UCSD and am getting some weird errors. I know next to nothing about this so am just following the instructions on Rich's website for using the MESA SDK (which has worked fine for my macbook many times). uname -m returns "x86_64" so I'm using the 64-bit linux version of the SDK. All the setup steps seem to work fine (no errors anyway), but I run into problems during the ./install step.
>>>>>>>>
>>>>>>>> The first error I encounter is:
>>>>>>>> gfortran: /lib64/libc.so.6: version `GLIBC_2.11' not found (required by gfortran)
>>>>>>>> make: *** [const_def.o] Error 1
>>>>>>>>
>>>>>>>> Perhaps tellingly, there is not a file named libc.so.6 in my mesa/utils/mesasdk/lib64.
>>>>>>>>
>>>>>>>> In addition, now whenever I try and use the man command I get the error:
>>>>>>>> /usr/bin/gtbl: error while loading shared libraries: /home/kmoore-ucsb/mesa_3794/utils/mesasdk/lib64/libstdc++.so.6: ELF file OS ABI invalid
>>>>>>>>
>>>>>>>> Any ideas what's going on from someone who may have done this before?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Kevin
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>>> _______________________________________________
>>>>>>>> mesa-users mailing list
>>>>>>>> mes...@li...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>>>
>>>>>>> ------------------------------
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> mesa-users mailing list
>>>>>>> mes...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>>>
>>>>>>>
>>>>>>> End of mesa-users Digest, Vol 29, Issue 2
>>>>>>> *****************************************
>>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>>> _______________________________________________
>>>>>> mesa-users mailing list
>>>>>> mes...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>>>>> infrastructure or vast IT resources to deliver seamless, secure access to
>>>>> virtual desktops. With this all-in-one solution, easily deploy virtual
>>>>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>>>>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>>>>> _______________________________________________
>>>>> mesa-users mailing list
>>>>> mes...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>>
>> ------------------------------------------------------------------------------
>> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
>> infrastructure or vast IT resources to deliver seamless, secure access to
>> virtual desktops. With this all-in-one solution, easily deploy virtual
>> desktops for less than the cost of PCs and save 60% on VDI infrastructure
>> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
>> _______________________________________________
>> mesa-users mailing list
>> mes...@li...
>> https://lists.sourceforge.net/lists/listinfo/mesa-users
>
>
> ------------------------------------------------------------------------------
> Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
> infrastructure or vast IT resources to deliver seamless, secure access to
> virtual desktops. With this all-in-one solution, easily deploy virtual
> desktops for less than the cost of PCs and save 60% on VDI infrastructure
> costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
> _______________________________________________
> mesa-users mailing list
> mes...@li...
> https://lists.sourceforge.net/lists/listinfo/mesa-users
|
