Intel's Cluster OpenMP(for distibuted memory)

[Tobias Burnus]

Hello,

as EXCITING currently only supports OpenMP parallization one cannot run it in parallel on distributed-memory clusters. Typical solution for distributed-memory systems is the more powerful MPI, but if I recall correctly (cannot find a reference at the moment) implementing MPI in EXCITING is not planed. And Cray's co-arrays, scheduled for the Fortran 2008 standard (cf. http://www.co-array.org/\) will not be soon in any compiler.

Therefore, you might be delighted that the Intel Fortran Compiler 9.1 (released yesterday) supports "Cluster OpenMP" on Itanium and em64t (= AMD64) systems. Behind the name coined by Intel is a slightly modified OpenMP which works though TCP or DAPL.

In the OpenMP model, most variables are global; however, with distributed memory this is not the case. Some variables are marked automatically as global variables ("shareable"), some may need to be marked manually. Depending on the OpenMP program, it may work immediately or not. (Opening a file in one process and writing to it from the other won't work.)
I have actually not yet tried Cluster OpenMP, especially not with EXCITING, but I plan to do so soon.

Tobias

PS: The Intel Compiler is free for non-commercial usage, cf. http://www.intel.com/cd/software/products/asmo-na/eng/compilers/219692.htm
Note, however:
"If you, as an individual, are receiving any form of compensation for the research project (i.e., you receive a salary, or funding, etc.), you do not qualify for a non-commercial use license."

[Tobias Burnus]

Forgot to add some references.

The whitepaper (how it is implemented) is available at:
http://www.intel.com/cd/software/products/asmo-na/eng/compilers/285865.htm

The more interesting documentation is the user manual, available at
http://softwareforums.intel.com/attachments/ids/11/3796/1/UsersGuide.pdf

Tobias

[Tobias Burnus]

Unfortunally, the non-commercial version of the Intel Fortran compiler 9.1 does not contain Cluster OpenMP.

Actually, Intel charges a lot for the Cluster version (too much for me/us at the moment):

Price for Intel Fortran compiler with only OpenMP:
- Non commercial version: $0
- Academic version: 279 Euros
- Commercial version: $699

Price for the Ifort with Cluster OpenMP
- Non commercial: Not Avail.
- Academic version: 3,471.88 Euros
- Commercial version: $9,999

Tobias

[Christian Meisenbichler]

as far as i understood intel's cluster OpenMp does not work with the default shared model like normal openmp but the user has to declare things as shared and issues synchronization commands to update local copies. It may still be working on standard OMP systems but standard OMP will for sure not work on cluster OMP. For real portability there is currently no alternative to MPI

[exciting]

Well that came out of the blue -- I hadn't realised OpenMP for clusters was so advanced!

If anyone manages to get EXCITING running with Cluster OpenMP, then let us know. I'm currently adding more OpenMP directives to the code, so any feedback at this stage would be useful.

Regards and thanks for the info,
Kay.