Read Me
This document contains information on how to build, install and run
the tests included in this package.
Prerequisites
==============
To build the examples you need the following installed in your
system,
* A C++ compiler that can build OpenCL programs and a fortran 90
compiler that can build OpenACC programs. We recommend using PGI
compilers (Refer to the PGI website for information on the version
of compilers that support OpenACC).
If you use a compiler that does not recognize OpenACC
(like GNU compilers for which support for OpenACC is under
development), applications that use OpenACC will not be built.
If you plan to build Intel MIC applications you need the Intel
Fortran compiler that supports the Intel MIC compiler directives
(Refer the Intel website for information on support for Intel
MIC pragmas)
Refer to the "Configure (Advanced)" section for more information
on how to use different (PGI and Intel) compilers to build the
different sets (OpenCL, OpenACC, Intel MIC) of applications.
* An MPI library (Use mpic++ and mpif90 to build the applications)
built with the above compilers
* An OpenCL SDK
* ESMF Library built using the compilers mentioned above. If you
want to build the suite using different compilers (different
compilers to build the different applications within the suite)
you need to build the ESMF library separately for each compiler.
Refer to "Configure (Advanced)" for some more information.
* Users can also choose to use custom Makefiles that include the
ESMF template makefiles (esmf.mk) to build the prototype applications
instead of the familiar configure/make/make install method. See
the section "Building Using Makefiles" below for more details.
Configure
==========
Run autoreconf to generate (or update) configuration files
bash:> autoreconf -fi
Set the CXX environment variable to point to mpic++ and the FC
environment variable to point to the mpif90 compiler wrappers.
Specify the path to OpenCL library using the "--with-opencl" option
and the path to ESMF library using the "--with-esmf" option. Add
paths to the ESMF dependent libraries to the LDFLAGS and add the
dependent libraries to the LIBS environment variable.
setenv CXX mpic++
setenv FC mpif90
setenv LDFLAGS "PATH_TO_ESMF_DEP_LIBS"
setenv LIBS "ESMF_DEP_LIBS"
./configure --prefix=/home/jayeshkrishna/espc/examples/install
--disable-shared --with-opencl=<OPENCL_INSTALL_DIR>
--with-esmf=<ESMF_INSTALL_DIR> |& tee configure.log
If you need to test ESMF Accelerator support (in an ESMF development
branch), pass "--enable-esmf-acc-support" to the configure script.
./configure --prefix=/home/jayeshkrishna/espc/examples/install
--disable-shared --with-opencl=<OPENCL_INSTALL_DIR>
--with-esmf=<ESMF_INSTALL_DIR>
--enable-esmf-acc-support |& tee configure.log
Configure (Advanced)
====================
The application suite consists of three different sets of
applications,
* OpenCL applications
* OpenACC applications
* Intel MIC applications
You can configure the suite to use a different compiler for each
set of the above applications.
CXX, FC => Is the default compilers used to build all applications.
This is also the set of compilers used to build OpenCL
applications. Use CXXFLAGS, FCFLAGS to set compiler
flags and LIBS to specify the libraries for linking.
OPENACC_FC => If set, is the compiler used to built all OpenACC
applications. Use OPENACC_FCFLAGS to set the
compiler flags and OPENACC_FCLIBS to specify
the libraries (dependencies) for linking with
the applications.
INTEL_MIC_FC => If set, is the compiler used to build all
Intel MIC applications. Use INTEL_MIC_FCFLAGS to
set the compilers flags and INTEL_MIC_FCLIBS to
specify the libraries to link with the applications.
Note that if OPENACC_FC (or INTEL_MIC_FC) is set FCFLAGS and LIBS
environment variables are no longer used for building OpenACC (or
Intel MIC) applications. As mentioned above use the corresponding
environment variables to specify the compiler flags and link libs.
Build and Install
==================
bash:> make |& tee make.log
bash:> make install |& tee install.log
Building using Makefiles
==========================================
The user can choose to build each application using a custom Makefile
instead of the familiar configure/make/make install method. If the user
chooses to do so, the user needs to use the Makefile.esmf* makefiles
included in the source to build the applications. For example, to build
Application 1 (opencl_info) the user needs to explicitly invoke make
with the Makefile.esmf makefile included with the source,
make -f Makefile.esmf
To use these makefiles the user needs to set the ESMFMKFILE environment
variable to point to the esmf.mk template makefile installed with ESMF
setenv ESMFMKFILE $ESMF_INSTALL_DIR/lib/esmf.mk
To build OpenCL applications the user also needs to set the path to
the OpenCL installation using the OPENCL_INSTALL_PATH environment
variable.
setenv OPENCL_INSTALL_PATH $OPENCL_INSTALL_DIR
Tests
=======
The tests included in the suite are,
* Application 1
This OpenCL (non-MPI) application provides information about the
accelerator devices attached to the system.
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/opencl_info.
Run the application as follows,
./opencl_info
* Application 2
This ESMF application contains three components,
** An MPI non-accelerated component
** An MPI accelerated component (The fortran stub of this component
calls an OpenCL C library routine to offload and execute a kernel
on a compute device)
** A Coupler
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/ESMF_OpenCL_simple_comp
Run the application using mpiexec as follows,
mpiexec -n 2 ./ESMF_OpenCL_simple_comp
* Application 3
This OpenACC (non-MPI) application provides information about the
accelerator devices attached to the system.
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/openacc_info
Run the application as follows,
./openacc_info
* Application 4
This ESMF application contains three components,
** An MPI non-accelerated component
** A component that uses OpenACC to accelerate some part of the code.
** A Coupler
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/ESMF_OpenACC_simple_app
Run the application using mpiexec as follows,
mpiexec -n 2 ./ESMF_OpenACC_simple_app
* Application 5
This ESMF application contains three components,
** An MPI non-accelerated component
** A component that uses OpenACC to accelerate some part of the code.
This component toggles (at runtime) between running the accelerated
part of code on the host and the accelerator.
** A Coupler
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/ESMF_OpenACC_toggle_app
Run the application using mpiexec as follows,
mpiexec -n 2 ./ESMF_OpenACC_toggle_app
* Application 6
This application (Intel-only) provides information about the
accelerator devices attached to the system.
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/intelmic_info
Run the application as follows,
./intelmic_info
* Application 7
This ESMF application contains three components,
** An MPI non-accelerated component
** A component that uses Intel MIC compiler directives to
accelerate part of the code. This component explicitly
offloads part of the computation to MICs attached to
the system
** A Coupler
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/ESMF_OpenACC_intelmic_expoff_app
Run the application using mpiexec as follows,
mpiexec -n 2 ./ESMF_IntelMIC_expoff_app
* Application 8
This ESMF application contains three components,
** A component that uses OpenCL to accelerate part of the code
execution. The fortran stub of this component calls an
OpenCL C library routine to offload and execute a kernel
on a compute device.
** A component that uses OpenACC compiler directives to
accelerate part of the code.
** A Coupler
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/ESMF_OpenACC_OpenCL_mixed_app
Run the application using mpiexec as follows,
mpiexec -n 2 ./ESMF_OpenACC_OpenCL_mixed_app
* Application 9
This ESMF application contains three components,
** A component that uses OpenCL to accelerate part of the code
execution. The fortran stub of this component calls an
OpenCL C library routine to offload and execute a kernel
on a compute device.
** A component that uses Intel MIC compiler directives to
accelerate part of the code. This component explicitly
offloads part of the computation to MICs attached to
the system
** A Coupler
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/ESMF_OpenCL_IntelMIC_mixed_app
Run the application using mpiexec as follows,
mpiexec -n 2 ./ESMF_OpenCL_IntelMIC_mixed_app
* Application 10
This ESMF application contains two components,
** An MPI non-accelerated component
** An MPI accelerated component (The fortran stub of this component
calls an OpenCL C library routine to offload and execute a kernel
on a compute device)
** A Coupler
This application uses the newly added ESMF_VMGet(...,
accDeviceCount=numAccDevices,...) interface to query the number
of accelerator devices. All the devices attached to a node are
allocated to the lowest ranked MPI process running on the node.
Pass "--enable-esmf-acc-support" to the configure script to
build this application (This application will not be built by
default)
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/ESMF_acc_support_OpenCL_app
Run the application using mpiexec as follows,
mpiexec -n 2 ./ESMF_acc_support_OpenCL_app
* Application 11
This ESMF application contains two components,
** An MPI non-accelerated component
** An MPI accelerated component (That uses OpenACC to
offload execution to accelerator devices)
** A Coupler
This application uses the newly added ESMF_VMGet(...,
accDeviceCount=numAccDevices,...) interface to query the number
of accelerator devices. All the devices attached to a node are
allocated to the lowest ranked MPI process running on the node.
Pass "--enable-esmf-acc-support" to the configure script to
build this application (This application will not be built by
default)
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/ESMF_acc_support_OpenACC_app
Run the application using mpiexec as follows,
mpiexec -n 2 ./ESMF_acc_support_OpenACC_app
* Application 12
This ESMF application contains two components,
** An MPI non-accelerated component
** An MPI accelerated component (That uses Intel MIC
to offload execution to accelerator devices)
** A Coupler
This application uses the newly added ESMF_VMGet(...,
accDeviceCount=numAccDevices,...) interface to query the number
of accelerator devices. All the devices attached to a node are
allocated to the lowest ranked MPI process running on the node.
Pass "--enable-esmf-acc-support" to the configure script to
build this application (This application will not be built by
default)
After the suite is installed, the application is available as
<INSTALL_DIR>/bin/ESMF_acc_support_IntelMIC_app
Run the application using mpiexec as follows,
mpiexec -n 2 ./ESMF_acc_support_IntelMIC_app