build-Supercom

Build guides for supercomputers

Grand Chariot, Hokkaido Univ.

module purge
module load intel/(version for the Intel compiler)
module load impi/(version for the Intel compiler)
./configure --with-mkl --prefix=$HOME/(directory to install the ERmod program)
make
make install

Cygnus, Tsukuba Univ.

module purge
module load openmpi/(version for the openmpi)/intel(version for the Intel compiler)-cuda(version for the cuda)
./configure --with-mkl --prefix=$HOME/(directory to install the ERmod program)
make
make install

For example, the line of "module load ..." is "module load openmpi/4.0.3/intel19.0.5-cuda10.1.2"

TSUBAME3.0, Tokyo Inst. Tech.

When compiled with the MKL library installed on the system

module load intel
module load intel-mpi
./configure --with-mkl --prefix=$HOME/(directory to install ERmod program)
make
make install

When compiled with the FFTW installed on the system

module load intel
module load intel-mpi
module load fftw
./configure --with-fftw --prefix=$HOME/(directory to install the ERmod program)
make
make install

Oakforest-PACS, JCAHPC (Tsukuba Univ. and Univ. Tokyo)

./configure --with-mkl --prefix=$HOME/(directory to install ERmod program)
make
make install

For the many-core usage, the program is built through

./configure --with-mkl --prefix=$HOME/(directory to install ERmod program) FCFLAGS="-axMIC-AVX512"
make
make install

Oakbridge-CX, Univ. Tokyo

module purge
module load intel/(version for the Intel compiler)
module load impi/(version for the Intel compiler)
./configure --with-mkl --prefix=$HOME/(directory to install the ERmod program)
make
make install

NEC LX, Institute for Molecular Sciences

./configure --with-mkl --prefix=$HOME/(directory to install the ERmod program)
make
make install

SQUID, Osaka Univ.

When compiled with the intel compiler,

module load BaseCPU/2024
./configure --with-mkl --prefix=$HOME/(directory to install the ERmod program)
make
make install

OCTOPUS, Osaka Univ.

When compiled with the intel compiler,

./configure --with-mkl --prefix=$HOME/(directory to install the ERmod program)
make
make install

When compiled with gfortran,

./configure FC=mpif90 --with-mkl --prefix=$HOME/(directory to install the ERmod program)
make
make install

ITO, Kyushu Univ.

module purge
module load intel/(version for the Intel compiler)
./configure --with-mkl --prefix=$HOME/(directory to install the ERmod program)
make
make install

Fugaku, RIKEN

# If conda is employed, comment out its activation lines (from "__conda_setup=..." to "unset __conda_setup") in the rc file  (e.g. ~/.bash_profile) while this installation. Otherwise, the conda's lib will be used.

# 1) update vmdplugins
cd vmdplugins
## edit Makefile as follows:
# From this position
CC = fccpx
CXX = FCCpx
SHLD = FCCpx -shared
CCFLAGS = -O -KPIC
CXXFLAGS = -O -KPIC
#NETCDF=yes      <-- insert '#' at the beginning of this line in Makefile
# To this position
## then compile, and copy the files to libexec
make
cp compile/dcdplugin.so libexec/           # necessary to read NAMD output
cp compile/gromacsplugin.so libexec/       # necessary to read gromacs output
cd ..

# 2) get FFTW directory path managed by public Spack
## source public Spack (The file path to be sourced may change. Check "Usage of OSSs" section in "Use of OSS" page on the Fugaku portal)
. /vol0004/apps/oss/spack/share/spack/setup-env.sh
## show the FFTW package (fujitsu-fftw) ID
spack find -lx
## load FFTW
spack load /(FFTW package ID)
## In the above, a space is there in front of the slash and is not after the slash
## get the installed directory path and see (directory to FFTW)/bin/fftw-wisdom
which fftw-wisdom

# 3) configure and compile ERmod
## FC value is automatically prefixed with "mpi".
./configure CC=mpifccpx CFLAGS=-Kfast FC=frtpx FCFLAGS=-Kfast --host=aarch64-linux-gnu --with-fftw=(directory to FFTW) --prefix=(directory to install ERmod program)
make
make install

If an error is encounter at step 3)

autoreconf -ivf
make
make install

Cray XC40, Kyoto Univ.

Two schemes are shown below, but the resulting ermod runs only serially.
A parallel run is not sucessful yet.

When compiled with the MKL library installed on the system

module switch PrgEnv-cray PrgEnv-intel
./configure --with-fft=mkl --prefix=$HOME/(directory to install ERmod program)
make
make install

When compiled with the FFTW installed on the system

module load fftw
module switch PrgEnv-cray PrgEnv-intel
./configure --with-fft=fftw --with-fftw=$FFTW_DIR/.. --prefix=$HOME/(directory to install ERmod program)
make
make install

Note in the above that $FFTW_DIR/.. refers to the directory above $FFTW_DIR, which is specified by module load fftw

Old Info

COMA, Tsukuba U

::::sh
module load intel/15.0.2
module load mkl/11.2.2
module load intelmpi/5.0.3
./configure --with-mkl --prefix=$HOME/(directory to install the ERmod program)
make
make install

CX400, Nagoya Univ.

When compiled with the MKL library installed on the system

source /center/local/apl/cx/intel/mkl/bin/mklvars.sh intel64
./configure --with-mkl --prefix=$HOME/(directory to install ERmod program)
make
make install

When user's shell is csh or tcsh, mklvars.sh within the above source command is to be changed into mklvars.csh

When compiled with the FFTW installed on the system

::::sh
./configure --with-fftw=/center/local/apl/cx/fftw-3.3.4 --prefix=$HOME/(directory to install ERmod program) CC=mpicc
make
make install

Cray CS400 2820XT, Kyoto Univ.

When compiled with the MKL library installed on the system

::::sh
./configure --with-fft=mkl --prefix=$HOME/(directory to install ERmod program)
make
make install

When compiled with the FFTW installed on the system

::::sh
module load fftw
./configure --prefix=$HOME/(directory to install ERmod program)
make
make install

Cray XC30, Kyoto U

To compile ERmod, it is necessary first to switch and add some modules as:

module switch craype-haswell craype-istanbul
module switch PrgEnv-cray PrgEnv-intel
module add fftw

The second step is to re-compile vmdplugins with:

cd vmdplugins
# edit Makefile as follows:
# From this position
CC = icc
CXX = icc
SHLD = icc -no-intel-extensions -shared -static-intel -static-libgcc
CCFLAGS =  -O -fPIC
CXXFLAGS = -O -fPIC
LDFLAGS= -Bstatic -lstdc++
# To this position
# then compile, and copy files to libexec
mkdir compile
make clean 
make dcdplugin.so gromacsplugin.so
cp compile/*.so libexec/
cd ..

It is further necessary to open vmdfio.c and modify a line from

::::C
handle = dlopen(pluginpath, RTLD_NOW | RTLD_GLOBAL);

to

::::C
handle = dlopen(pluginpath, RTLD_LAZY);

If the modification of vmdfio.c is not done , the compilation still works but a segmentation error appears at runtime.
Some environment variables are then set, and configure, make, & make install are done.

::::sh
export CC=cc
export FC=ftn
export MPIFC=ftn

./configure --prefix=(path to install ERmod program)
make
make install

Cray XE6, Yokohama City U. and Kyoto U.

Dr. Ikeguchi reported that the following scheme is necessary.
The first is to recompile vmdplugins with:

cd vmdplugins
# edit Makefile as follows:
# From this position
CC = icc
CXX = icc
SHLD = icc -no-intel-extensions -shared -static-intel -static-libgcc
CCFLAGS =  -O -fPIC
CXXFLAGS = -O -fPIC
LDFLAGS= -Bstatic -lstdc++
# To this position
# then compile, and copy files to libexec
mkdir compile
make clean 
make dcdplugin.so gromacsplugin.so
cp compile/*.so libexec/
cd ..

The second step is to open vmdfio.c and modify a line from

::::C
handle = dlopen(pluginpath, RTLD_NOW | RTLD_GLOBAL);

to

::::C
handle = dlopen(pluginpath, RTLD_LAZY);

A few modules are then read, some environment variables are set, and configure, make, & make install are done.

::::sh
module load PrgEnv-intel
module load fftw

export CC=cc
export F77=ftn
export FC=ftn
export MPIFC=ftn

./configure --prefix=(path to install ERmod program)
make
make install

FX10

::::sh
# Compile FFTW
module load fftw/3.3
# If FFTW is to be compiled by user
# ./configure CC=fccpx CFLAGS=-Kfast --enable-fma --host=sparc64-unknown-linux-gnu --prefix=$HOME/(directory to which FFTW is installed)
# make
# make install

# Compile ERmod
# update vmdplugins first
cd vmdplugins
# edit Makefile as follows:
# From this position
CC = fccpx
CXX = FCCpx
SHLD = FCCpx -shared
CCFLAGS = -O -KPIC
CXXFLAGS = -O -KPIC
#NETCDF=yes      <-- insert '#' at the beginning of this line in Makefile
# To this position
# then compile, and copy files to libexec
make
cp compile/dcdplugin.so libexec/           # necessary to read NAMD output
cp compile/gromacsplugin.so libexec/    # necessary to read gromacs output
cd ..

# Configure and compile ERmod
./configure CC=mpifccpx CFLAGS=-Kfast,parallel FC=mpifrtpx FCFLAGS=-Kfast,parallel --host=sparc64-unknown-linux-gnu --with-fft=fftw --prefix=$HOME/(directory to install ERmod program)
# If user has installed his own FFTW,
# ./configure CC=mpifccpx CFLAGS=-Kfast,parallel FC=mpifrtpx FCFLAGS=-Kfast,parallel --host=sparc64-unknown-linux-gnu --with-fftw=$HOME/(directory to which FFTW is installed) --prefix=$HOME/(directory to install ERmod program)
make
make install

It should be emphasized that the plugin needs to be compiled, too.

K, RIKEN

The program can be compiled with a procedure similar to the one described above for FX10.

::::sh
# Compile ERmod
# update vmdplugins first
cd vmdplugins
# edit Makefile as follows:
# From this position
CC = fccpx
CXX = FCCpx
SHLD = FCCpx -shared
CCFLAGS = -O -KPIC
CXXFLAGS = -O -KPIC
#NETCDF=yes      <-- insert '#' at the beginning of this line in Makefile
# To this position
# then compile, and copy files to libexec
make
cp compile/dcdplugin.so libexec/           # necessary to read NAMD output
cp compile/gromacsplugin.so libexec/    # necessary to read gromacs output
cd ..

# Configure and compile ERmod
./configure CC=mpifccpx CFLAGS=-Kfast,parallel FC=mpifrtpx FCFLAGS=-Kfast,parallel --host=sparc64-unknown-linux-gnu --with-fftw=/home/apps/fftw/3.3.3 --prefix=$HOME/(directory to install ERmod program)
make
make install

It is not necessary to load or install FFTW, and it is necessary to specifiy the directory where FFTW is installed.

A rather specific script is needed for running ermod in connection to the process of staging.
An example script for a soln calculation reads

::::sh
#/bin/bash -x
#PJM --rsc-list "node=64"
#PJM --rsc-list "elapse=24:00:00"
#PJM --name soln
#PJM --mpi "shape=64"
#PJM --mpi "proc=512"
#PJM -s
#
#PJM --stg-transfiles all
#
#PJM --vset path=(directory in which the solution MD is executed)
#PJM --vset traj=solution_run
#PJM --stgin "(directory in which ERmod is installed)/ermod-libexec.tar.gz ./"
#PJM --stgin "(directory in which ERmod is installed)/bin/ermod ./"
#
#PJM --stgin "${path}${traj}.xtc ./"
#PJM --stgin "MDinfo ./"
#PJM --stgin "MolPrm1 ./"
#PJM --stgin "SltInfo ./"
#PJM --stgin "parameters_er ./"
#
#PJM --stgout "*.tt ./"
#PJM --stgout "engsln.* ./"
#PJM --stgout "weight_soln ./"

. /work/system/Env_base
#
ln -sf solution_run.xtc HISTORY  # the filename defined above as traj needs to be used
tar xzf ermod-libexec.tar.gz
export ERMOD_PLUGINS="libexec/ermod"
mpiexec ./ermod

where the lines starting with #PJM are not comments. They are part of the script.

An example script for a refs calculation reads

::::sh
#/bin/bash -x
#PJM --rsc-list "node=64"
#PJM --rsc-list "elapse=24:00:00"
#PJM --name refs
#PJM --mpi "shape=64"
#PJM --mpi "proc=512"
#PJM -s
#
#PJM --stg-transfiles all
#
#PJM --vset path_v=(directory in which the solvent MD is executed)
#PJM --vset traj_v=solvent_run
#PJM --vset path_u=(directory in which the MD of isolated solute is executed)
#PJM --vset traj_u=solute_run
#PJM --stgin "(directory in which ERmod is installed)/ermod-libexec.tar.gz ./"
#PJM --stgin "(directory in which ERmod is installed)/bin/ermod ./"
#
#PJM --stgin "${path_v}${traj_v}.xtc ./"
#PJM --stgin "${path_u}${traj_u}.xtc ./"
#
#PJM --stgin "MDinfo ./"
#PJM --stgin "MolPrm1 ./"
#PJM --stgin "SltInfo ./"
#PJM --stgin "parameters_er ./"
#
#PJM --stgout "*.tt ./"
#PJM --stgout "engref.* ./"
#PJM --stgout "corref.* ./"
#PJM --stgout "weight_refs ./"

. /work/system/Env_base
#
ln -sf solvent_run.xtc HISTORY  # the filename defined above as traj_v needs to be used
ln -sf solute_run.xtc SltConf  # the filename defined above as traj_u needs to be used
tar xzf ermod-libexec.tar.gz
export ERMOD_PLUGINS="libexec/ermod"
mpiexec ./ermod

where the lines starting with #PJM are not comments. They are part of the script.

An example script for a slvfe run reads

::::sh
#!/bin/sh -x
#PJM --rsc-list "node=1"
#PJM --rsc-list "elapse=01:00:00"
#PJM --name slvf
#PJM -s
#PJM --stg-transfiles all
#PJM --vset path=(directory in which slvfe runs)
#PJM --stgin "${path}soln.tar.gz ./"
#PJM --stgin "${path}refs.tar.gz ./"
#PJM --stgin "(directory in which ERmod is installed)/bin/slvfe ./"
#PJM --stgin "parameters_fe ./"
#PJM -o "slvfe.txt"

. /work/system/Env_base
tar xzf ermod-libexec.tar.gz
tar xzf soln.tar.gz
tar xzf refs.tar.gz
export ERMOD_PLUGINS="libexec/ermod"
./slvfe

where the lines starting with #PJM are not comments. They are part of the script.

In the above, ermod-libexec.tar.gz is the compressed file for the libexec directory within the directory into which the programs are installed with make install.
It is further noted that the HISTORY file generated by gen_input is not staged since it is a symbolic link.
It is thus necessary to have a line of symbolic link of the trajectory data file to HISTORY.

The points are; 1) a symbolic link of HISTORY to a trajectory file is necessary at the staged directory and 2) ERMOD_PLUGINS is set to the directory containing the .so files.

Actually, the ermod program searches the VMD plugin first into the directory set by ERMOD_PLUGINS and then into the libexec directory within the directory specified when make install is carried out.
When K is not used (no staging), the second search is usually sufficient for detecting the plugin files.
The staging is conducted on K, though, and the directory specified at compilation is lost.
It is necessary to manually fix the ERMOD_PLUGINS variable.

FX1

::::sh
# FFTW
CC=fcc ./configure --prefix=$HOME/(directory to which FFTW is installed) --enable-fma
make -j4
make install
# ermod
./configure FCFLAGS="-Adm" --with-blas=-SSL2 FC=mpifrt F77=mpifrt CC=fcc --prefix=./configure --with-mkl --prefix=$HOME/(directory to install ERmod program) --with-fftw=$HOME/(directory to which FFTW is installed)
make
# manually link programs because FCFLAGS conflicts

vmdplugins/Makefile:

::::Makefile
CC = fcc
CXX = FCC
SHLD = FCC -G -Bdynamic -Ktl_notrt -lstl
CCFLAGS = -O -g -KPIC
CXXFLAGS = -O -g -KPIC
#NETCDF=no
NETCDFLIB=-lnetcdf

Altix, IMS

::::sh
source /opt/intel/Compiler/11.1/075/bin/ifortvars.csh ia64
cd vmdplugin
make
cp compile/*.so libexec
./configure --with-mkl --prefix=$HOME/(directory to install ERmod program) 
make
make install

Primequest, IMS

::::sh
source /opt/intel/Compiler/11.1/075/bin/ifortvars.csh ia64
cd vmdplugin
make
cp compile/*.so libexec
./configure --with-mkl --prefix=$HOME/(directory to install ERmod program) 
make
make install

SR-16000, IMS

::::sh
export LANG=C # bash
# setenv LANG C # for csh or tcsh

# FFTW
./configure AR="ar -X64" --enable-fma
# ermod
./configure FFLAGS="-qfixed=132 -qrealsize=8" FCFLAGS="-qrealsize=8" --with-fftw=$HOME --with-blas=...
(TODO)

T2K

::::sh
source /opt/itc/mpi/mpiswitch.sh mpich-mx-intel11
source /opt/intel/Compiler/11.0/074/mkl/tools/environment/mklvarsem64t.sh
FFLAGS=-msse3 FCFLAGS=-msse3 ./configure --with-mkl --prefix=$HOME/(directory to install ERmod program)
make
make install

Altix (System B) @ ISSP, U of Tokyo

::::sh
source /opt/intel/Compiler/12.0/084/bin/compilervars.sh intel64
./configure --with-mkl --prefix=$HOME/(directory to install ERmod program)
make
make install