From: John P. <pet...@cf...> - 2006-11-07 19:49:05
Attachments:
Makefile
extract_make_var.sh
|
Hi, I'm following up to my own previous posting about PETSc's configuration files and the difficulties of including them directly in LibMesh's Makefile. What I came up with (thanks also to Roy!) is a simple shell script which will process the PETSc configuration files separately (i.e. no more including the petscconf/packages/base.site file directly. Yes, its name really has changed that many times since 2.1.0.) As an added bonus to having a cleaner Makefile namespace, this should help streamline the linking process as well. Could you all do me a favor and try running the following sample Makefile on your systems (assuming you have PETSc installed)? Just copy the attached sample Makefile and shell script into any directory, chmod a+x the script and type make. Let me know if there are any error messages or strange output, it should just spit out the link line, include directories, and compiler flags specified in $(PETSC_DIR)/bmake/common/variables. Thanks, John |
From: <ti...@ce...> - 2006-11-08 07:24:42
|
Dear John, On Tue, 7 Nov 2006, John Peterson wrote: > Could you all do me a favor and try running the following sample > Makefile on your systems (assuming you have PETSc installed)? > Just copy the attached sample Makefile and shell script into any > directory, chmod a+x the script and type make. Let me > know if there are any error messages or strange output, it should > just spit out the link line, include directories, and compiler > flags specified in $(PETSC_DIR)/bmake/common/variables. For me, it doesn't work unless I prefix the calls to the script with "./", because "." is not in my $PATH. If I do so, it works as expected. Best Regards, Tim |
From: John P. <pet...@cf...> - 2006-11-08 14:45:12
|
Tim Kr=F6ger writes: > Dear John, >=20 > On Tue, 7 Nov 2006, John Peterson wrote: >=20 > > Could you all do me a favor and try running the following sample > > Makefile on your systems (assuming you have PETSc installed)=3F > > Just copy the attached sample Makefile and shell script into any > > directory, chmod a+x the script and type make. Let me > > know if there are any error messages or strange output, it should > > just spit out the link line, include directories, and compiler > > flags specified in $(PETSC=5FDIR)/bmake/common/variables. >=20 > For me, it doesn't work unless I prefix the calls to the script with= =20 > "./", because "." is not in my $PATH. If I do so, it works as=20 > expected. Excellent point, I probably shouldn't have "." in my path either, it's just so damn convenient :) Thanks for your comments, J |
From: Steffen P. <ste...@tu...> - 2006-11-08 12:00:50
|
It works fine on our system. Steffen > Hi, > > I'm following up to my own previous posting about PETSc's > configuration files and the difficulties of including them > directly in LibMesh's Makefile. > > What I came up with (thanks also to Roy!) is a simple shell script > which will process the PETSc configuration files separately (i.e. no > more including the petscconf/packages/base.site file directly. Yes, > its name really has changed that many times since 2.1.0.) > As an added bonus to having a cleaner Makefile namespace, this should > help streamline the linking process as well. > > Could you all do me a favor and try running the following sample > Makefile on your systems (assuming you have PETSc installed)? > Just copy the attached sample Makefile and shell script into any > directory, chmod a+x the script and type make. Let me > know if there are any error messages or strange output, it should > just spit out the link line, include directories, and compiler > flags specified in $(PETSC_DIR)/bmake/common/variables. > > Thanks, > John > > |
From: Karl T. <k.t...@au...> - 2006-11-08 22:10:56
|
John Peterson writes: > I'm following up to my own previous posting about PETSc's > configuration files and the difficulties of including them > directly in LibMesh's Makefile. Hi John. Thanks for looking into this. I'd been trying to find a good way to do this too because of unresolved mpe symbols as discussed in http://sourceforge.net/mailarchive/message.php?msg_id=37104990 > > What I came up with (thanks also to Roy!) is a simple shell script > which will process the PETSc configuration files separately (i.e. no > more including the petscconf/packages/base.site file directly. Yes, > its name really has changed that many times since 2.1.0.) > As an added bonus to having a cleaner Makefile namespace, this should > help streamline the linking process as well. Seems like the right sort of thing to do. Note that for petsc-2.1.5 and older, bmake/common/base already has getlinklibs and getincludedir targets, but 2.1.3 does not have these. I don't know whether LibMesh should still support 2.1.0. There is nothing corresponding to PETSC_COMPILE but I don't know what needs this. > > Could you all do me a favor and try running the following sample > Makefile on your systems (assuming you have PETSc installed)? > Just copy the attached sample Makefile and shell script into any > directory, chmod a+x the script and type make. Let me > know if there are any error messages or strange output, it should > just spit out the link line, include directories, and compiler > flags specified in $(PETSC_DIR)/bmake/common/variables. It works most of the time, but, when it is part of a recursive make, there are problems: /bin/sh: command substitution: line 0: unexpected EOF while looking for matching `'' /bin/sh: command substitution: line 1: syntax error: unexpected end of file PETSC_LIB=make[1]: Entering directory /home/karl/tmp/recursive-make' ... This can be reproduced by placing the files in a subdirectory (called recursive-make here) and running make in the parent directory on a Makefile with contents: all: make -C recursive-make I had suggested make --no-print-directory to work around this, but this is likely to cause problems for non-GNU make commands. It seems to me that these things should be worked out at configure (not make) time and so recursive makes would not be very likely. But a possible workaround is to include the following line in extract_make_var.sh before invoking make: unset MAKELEVEL HTH, Karl. |
From: Roy S. <roy...@ic...> - 2006-11-08 22:23:37
|
On Thu, 9 Nov 2006, Karl Tomlinson wrote: > It seems to me that these things should be worked out at configure > (not make) time and so recursive makes would not be very likely. That's right; we don't want every process of every parallel make to start doing funny variable tricks - for one thing, that script uses a fixed temp file name. We could always fix that by using mktemp, but how many other weird special cases haven't we thought of? The safest solution has got to be to have ./configure write out the correct PETSc variables once so we don't have to go looking them up again. --- Roy |
From: John P. <pet...@cf...> - 2006-11-08 23:14:57
|
Roy Stogner writes: > On Thu, 9 Nov 2006, Karl Tomlinson wrote: > > > It seems to me that these things should be worked out at configure > > (not make) time and so recursive makes would not be very likely. I definitely agree with you: configure is the right time to set these variables! The truly prodigious PETSC_LIB variable from the Texas computing center is pasted below, it has over 300 lines and 15,000 characters (most of which is repeated -Wl,-rpath directives...). Actually I am impressed that the linker can handle it at all. This would probably be tough to read in a LibMesh Makefile, but if the alternative is breaking people's recursive makes, then this is the way to go. Maybe 'include'ing a Makefile.petsc with this variable set would be a good alternative. -John -Wl,-rpath,/opt/apps/petsc-intel9-2.3.2/2.3.2/lib/em64t -L/opt/apps/petsc-intel9-2.3.2/2.3.2/lib/em64t -lpetscts -lpetscsnes -lpetscksp -lpetscdm -lpetscmat -lpetscvec -lpetsc /opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/spooles-2.2/em64t/MPI/src/spoolesMPI.a /opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/spooles-2.2/em64t/spooles.a -Wl,-rpath,/opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/hypre-1.11.1b/em64t/lib -L/opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/hypre-1.11.1b/em64t/lib -lHYPRE_DistributedMatrix -lHYPRE_DistributedMatrixPilutSolver -lHYPRE_Euclid -lHYPRE_IJ_mv -lHYPRE_LSI -lHYPRE_MatrixMatrix -lHYPRE_ParaSails -lHYPRE_krylov -lHYPRE_parcsr_ls -lHYPRE_parcsr_mv -lHYPRE_seq_mv -lHYPRE_sstruct_ls -lHYPRE_sstruct_mv -lHYPRE_struct_ls -lHYPRE_struct_mv -lstdc++ -lpmpich++ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -lcxaguard -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/apps/pmetis/pmetis-3.1 -L/opt/apps/pmetis/pmetis-3.1 -lparmetis -Wl,-rpath,/opt/apps/pmetis/pmetis-3.1 -L/opt/apps/pmetis/pmetis-3.1 -lmetis -Wl,-rpath,/opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/ml/em64t/lib -L/opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/ml/em64t/lib -lml -lstdc++ -lpmpich++ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -lcxaguard -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/spai_3.0-mar-06/em64t/lib -L/opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/spai_3.0-mar-06/em64t/lib -lspai -Wl,-rpath,/opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/Chaco-2.2/em64t/lib -L/opt/apps/petsc-intel9-2.3.2/2.3.2/externalpackages/Chaco-2.2/em64t/lib -lchaco -Wl,-rpath,/opt/apps/gotoblas/gotoblas-1.02 -L/opt/apps/gotoblas/gotoblas-1.02 -lgoto -Wl,-rpath,/opt/intel/mkl8.1/lib/em64t -L/opt/intel/mkl8.1/lib/em64t -lmkl_lapack64 -lmkl -lguide -lpthread -Wl,-rpath,/opt/apps/gotoblas/gotoblas-1.02 -L/opt/apps/gotoblas/gotoblas-1.02 -lgoto -Wl,-rpath,/opt/ofed//lib64 -L/opt/ofed//lib64 -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -L/opt/intel/compiler9.1/fc/lib/ -lifcore -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -L/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -L/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -ldl -lpmpich -lmpich -Wl,-rpath,/opt/ofed//lib64/ -L/opt/ofed//lib64/ -libverbs -lpthread -lrt -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -L/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -L/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -lsvml -limf -lipgo -lirc -lgcc_s -lirc_s -lmpichf90nc -lmpichfarg -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc//lib -L/opt/intel/compiler9.1/fc//lib -lifport -lm -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -lm -Wl,-rpath,/opt/ofed//lib64 -L/opt/ofed//lib64 -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -L/opt/intel/compiler9.1/fc/lib/ -lifcore -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -L/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -L/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -ldl -lpmpich -lmpich -Wl,-rpath,/opt/ofed//lib64/ -L/opt/ofed//lib64/ -libverbs -lpthread -lrt -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -L/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -L/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -lsvml -limf -lipgo -lirc -lgcc_s -lirc_s -lmpichf90nc -lmpichfarg -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc//lib -L/opt/intel/compiler9.1/fc//lib -lifport -lm -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -lm -lstdc++ -lpmpich++ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -lcxaguard -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -lstdc++ -lpmpich++ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -lcxaguard -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -Wl,-rpath,/opt/ofed//lib64/ -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -L/opt/intel/compiler9.1/fc/lib/ -lifcore -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -L/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -L/opt/MPI/intel9/mvapich-gen2/0.9.8/lib -ldl -lpmpich -lmpich -Wl,-rpath,/opt/ofed//lib64/ -L/opt/ofed//lib64/ -libverbs -lpthread -lrt -Wl,-rpath,/opt/ofed//lib64 -L/opt/ofed//lib64 -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc//lib -L/opt/intel/compiler9.1/cc//lib -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -L/usr/lib/gcc/x86_64-redhat-linux/3.4.4/ -Wl,-rpath,/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/3.4.4/../../../../lib64 -lsvml -limf -lipgo -lirc -lgcc_s -lirc_s -Wl,-rpath,/opt/ofed//lib64 -Wl,-rpath,/opt/MPI/intel9/mvapich-gen2/0.9.8/lib/shared -Wl,-rpath,/opt/intel/compiler9.1/fc/lib/ -Wl,-rpath,/opt/intel/compiler9.1/cc/lib/ -ldl |