GALPROP README
---------------------------------------------------------------------
1. GALPROP is a code for cosmic ray propagation and interstellar gamma-ray and synchrotron radiation.
****************************************************************************************
For Documentation, see http://www.mpe.mpg.de/~aws/galprop_explanatory_supplement.pdf
which is also included in this distribution.
*****************************************************************************************
2. Installation
The packages cfitsio, CCfits, healpix and gsl (GNU Scientific Library) and CLHEP are required.
These can be obtained from the following URLs
cfitsio: http://heasarc.gsfc.nasa.gov/fitsio
CCfits: http://heasarc.gsfc.nasa.gov/fitsio
healpix: http://healpix.jpl.nasa.gov This version requires HEALPix 2.20a or 3.11 (may need openmp enabled in configure, see below)
gsl: http://www.gnu.org/software/gsl/
CLHEP: http://wwwasd.web.cern.ch/wwwasd/lhc++/clhep
Typical installation goes like this:
cd <directory where working copy of galprop and this README file is>
autoreconf -i -v -f
./configure --with-cfitsio=<path for cfitsio> --with-ccfits=<path for ccfits> --with-healpix=<path for healpix>/src/cxx/generic_gcc --with-gsl=<path for gsl> --with-clhep=<path for CLHEP> --prefix=<path for installed software> --with-galdef=<path for GALDEF files> --with-fitsdata=<path for FITS files> --datadir=<path for .dat files> --enable-openmp
make
which generates the binary ./source/galprop
then (NB from r304 not any more optional, but required)
make install
which puts the binary in <path for installed software>/bin
and the library in <path for installed software>/lib
and the headers in <path for installed software>/include
and copies the .dat files needed by galprop to <path for .dat files>
Note that with this config method, LD_LIBRARY_LIST is not required since the information on where to find the libraries is in the executable.
20090721: from r273 DO need setenv LD_LIBRARY_LIST <path for CLHEP libraries > (Hopefully will eventually treat this as for the other libraries)
General rule: if only static library (.a) is available this is used, adding to LD_LIBRARY_PATH is not required.
if only runtime library (.so) is available this is used, adding to LD_LIBRARY_PATH is required.
If both static and runtime libraries are available, it varies from case to case. gsl, CCfits: uses static library. CLHEP: uses runtime library.
The code is tested on a variety of compilers and linux distributions. In
addition, the code autodetects and compiles on the Apple/Mac version of the
BSD. The code will probably autodetect and compile under other
Unix-like variants this is untested. We don't support, nor care, about a
windows-based environment -- requests for such support will be ignored!
(documented by AWS, updated 20090612: added ccfits to config, added note on LD_LIBRARY_LIST)
3. Some explanation of how the autotools installation works
The (only) installation files in the repository are these 7:
configure.ac acinclude.m4
./Makefile.am ./Makefile.am.common ./Makefile.am.common-tail ./source/Makefile.am
galprop-config.in
Then...
configure.ac acinclude.m4 are read by autoreconf to produce the shell script ./configure (and various other files)
while
./Makefile.am ./Makefile.am.common ./Makefile.am.common-tail ./source/Makefile.am
are read by ./config to generate ./Makefile.in ./Makefile ./source/Makefile.in ./source/Makefile and other files like ./libtools
>> The actual routines to be processed are specified in ./source/Makefile.am
./Makefile.am just points to source/Makefile.am (it works recursively)
4. The current GALPROP version is 54 (starting at svn revision 151: before that is version 53)
5. To run galprop
create directory <path for GALDEF files> and edit your galdef_54_xxxxxxx (start from examples in ./GALDEF)
create directory <path for FITS files> then copy the required input files there from ./FITS (see galdef for names of required files)
create directory <path for output FITS files> if different path required from <path for FITS files>. Now controlled on command line.
(now controlled by .configure, from r293)
then e.g. to process galdef_54_610202h7S :
> <directory for installed software>/bin/galprop -r 610202h7S -o <path for output FITS files>
the output goes to <path for output FITS files>, defaults to ./configure <path for FITS files>, or ../FITS if not specified in ./configure
6. An example configuration
Here is how it works at MPE at r324
Note the additional parameters for GALDEF and FITS files and fortran_data, and also --prefix for the installed software.
If --with-fitsdata is absent, defaults to ../FITS
Note that from r324 the optimization option (CXXFLAGS) is obsolete, and is provided automatically.
HEALPix 3.11:
./configure --with-cfitsio=/afs/ipp-garching.mpg.de/home/a/aws/propagate/c/cfitsio/3.26/gcc_sles11_olga2/cfitsio --with-ccfits=/afs/ipp-garching.mpg.de/home/a/aws/propagate/c/CCfits/gcc_olga2_sles11_cfitsio3.26/usr/local --with-healpix=/afs/ipp-garching.mpg.de/home/a/aws/Healpix/Healpix_3.11/src/cxx/generic_gcc --with-gsl=/afs/ipp-garching.mpg.de/home/a/aws/gsl/gsl-1.10/olga --with-clhep=/afs/ipp-garching.mpg.de/home/a/aws/propagate/c/CLHEP/2.0.4.3/installed_here --with-galdef=/afs/ipp/u/aws/propagate/c/GALDEF --with-fitsdata=/afs/ipp/u/aws/propagate/c/FITS --datadir=/afs/ipp/u/aws/propagate/c/galprop/fortran_data --enable-openmp
... with intel compiler:
./configure --with-cfitsio=/afs/ipp-garching.mpg.de/home/a/aws/propagate/c/cfitsio/3.26/gcc_sles11_olga2/cfitsio --with-ccfits=/afs/ipp-garching.mpg.de/home/a/aws/propagate/c/CCfits/gcc_olga2_sles11_cfitsio3.26/usr/local --with-healpix=/afs/ipp-garching.mpg.de/home/a/aws/Healpix/Healpix_3.11/src/cxx/generic_gcc --with-gsl=/afs/ipp-garching.mpg.de/home/a/aws/gsl/gsl-1.10/olga --with-clhep=/afs/ipp-garching.mpg.de/home/a/aws/propagate/c/CLHEP/2.0.4.3/installed_here --with-galdef=/afs/ipp/u/aws/propagate/c/GALDEF --with-fitsdata=/afs/ipp/u/aws/propagate/c/FITS --datadir=/afs/ipp/u/aws/propagate/c/galprop/fortran_data --enable-openmp CXX=icpc F77=ifort FC=ifort
HEALPix 2.20a:
./configure --with-cfitsio=/afs/ipp-garching.mpg.de/home/a/aws/propagate/c/cfitsio/3.26/gcc_sles11_olga2/cfitsio --with-ccfits=/afs/ipp-garching.mpg.de/home/a/aws/propagate/c/CCfits/gcc_olga2_sles11_cfitsio3.26/usr/local --with-healpix=/afs/ipp-garching.mpg.de/home/a/aws/Healpix/Healpix_2.20a_sles11_olga2/Healpix_2.20a/src/cxx/generic_gcc --with-gsl=/afs/ipp-garching.mpg.de/home/a/aws/gsl/gsl-1.10/olga --with-clhep=/afs/ipp-garching.mpg.de/home/a/aws/propagate/c/CLHEP/2.0.4.3/installed_here --with-galdef=/afs/ipp/u/aws/propagate/c/GALDEF --with-fitsdata=/afs/ipp/u/aws/propagate/c/FITS --datadir=/afs/ipp/u/aws/propagate/c/galprop/fortran_data --enable-openmp
make
make install
setenv LD_LIBRARY_PATH /afs/ipp-garching.mpg.de/home/a/aws/propagate/c/CLHEP/2.0.4.3/installed_here/lib
cd <anywhere you like>
/afs/ipp/u/aws/propagate/c/galprop/installed/bin/galprop -r reltest01 -o <path for output FITS files>
and watch it run.
there are (from r315) additional command line options. Here are all of them as at r324:
-r run ID mandatory
-o <path for output FITS files) overrides ./configure value
-g <path for GALDEF files> overrides ./configure value
-f <path for input FITS files> overrides ./configure value
-p <prefix> optional prefix for the output files
-h shows galprop usage help
If using OpenMP the number of threads can be controlled with e.g.
setenv OMP_NUM_THREADS 8
if using intel compiler you may also need something like
setenv LD_LIBRARY_PATH ${LD_LIBRARY_PATH}:/afs/@cell/common/soft/intel/Compiler/11.1/c/lib/intel64
To redirect output from the screen to a file, ">" is not sufficient, instead do e.g.:
source/galprop -r reltest01 >& logfile
7. Test cases
Test cases are provided, which exercise most of the functionality of galprop in 2D and 3D.
They use a very coarse grid in space and energy so that they run in a few minutes CPU and little memory.
They can be used to test the installation (for accurate results a much finer grid should be used, taking correspondingly more CPU/memory).
> cp GALDEF/galdef_54_reltest01 <path for GALDEF files> (2D, skymap format = 3 = healpix)
> cp GALDEF/galdef_54_reltest02 <path for GALDEF files> (3D, skymap format = 3 = healpix)
> cp GALDEF/galdef_54_reltest03 <path for GALDEF files> (2D, skymap format = 0 = l,b )
> cp GALDEF/galdef_54_reltest04 <path for GALDEF files> (3D, new ISRF, only to r422, will not work with later revisions
> cp GALDEF/galdef_54_reltest05 <path for GALDEF files> (2D, new aniso ISRF, iso. IC, from r423)
> cp GALDEF/galdef_54_reltest06 <path for GALDEF files> (2D, new aniso ISRF, aniso. IC, from r423, OMP compilation only)
> cp GALDEF/galdef_54_reltest09 <path for GALDEF files> (2D, as reltest01 but explicit propagation method)
> cp GALDEF/galdef_54_reltest10 <path for GALDEF files> (3D, synchrotron including Stokes for a 3D B-field model, free-free absorption and emission)
> cp GALDEF/galdef_54_reltest11 <path for GALDEF files> (2D, convection=2,3 : new formulation)
> cp GALDEF/galdef_54_reltest13 <path for GALDEF files> (2D, anisotropic diffusion)
> cp GALDEF/galdef_54_reltest14 <path for GALDEF files> (3D, anisotropic diffusion)
> <path for installed software>/bin/galprop -r reltest01 -o <path for output FITS files>
> <path for installed software>/bin/galprop -r reltest02 -o <path for output FITS files>
8. Other configure options
In addition to the standard options listed above, configure accepts:
--enable-debug compiles with debug flags for use with gdb
--enable-openmp compiles openmp directives into code
CXX=<c++ compiler> e.g. icpc for intel compiler (to replace default gnu compiler)
F77=<fortran compiler> e.g. ifort for intel compiler (to replace default gun compiler). NB g77 is not supported. gfortran is supported: if not default then use F77=gfortran, otherwise ./configure may complain about openmp being not supported.
FC=<fortran compiler> e.g. ifort for intel compiler (to replace default gun compiler). Needed for two f90 hadronic production routines.
Note: the code is built by default with optimisation options -O3 and SSE
vector instructions (on x86 machines -- other architectures have similar
options but have not been tuned due to no access to machines). We are working
to provide more tuning dependent on processor sub-type and compiler. We test
on multiple versions of gcc and intel compilers (currently we regularly build
using gcc/gfortran 4.1.2, 4.3.4, and 4.4.1, and icpc/ifort 10 and 11 with
scientific, suse, and gentoo linux). The best optimisation options are
currently provided for later gccs (4.3 and 4.4) and the intel compiler,
mainly because those are used by the one (TAP) testing and tuning the
auto-configuration uses in a production environment.
9. ISRF files.
from around r315: use
ISRF_filetype = 2 or 3. 1 is now obsolete, while 2 specifies what was formerly 1
(so galdef files are not upwards compatible, and 1 will exit with error message !).
ISRF_filetype = 3 for the new anisotropic ISRF. ISRF_file = <location of ISRF dat file under FITS directory>
e.g. ISRF_file = Standard30kpc/StandardHalo30kpc.dat
for ISRF data at <path for FITS files>/Standard30kpc with dat file at <path for FITS files>/Standard30kpc/StandardHalo30kpc.dat
from r423 ISRF_filetype = 3 must use a new set of ISRFs: (the above type 3 will no longer work!)
ISRF_file = ISRF/Standard/Standard.dat
this set of files (~1 GB ! is included in the svn distribution of r423), under FITS/ISRF: copy this directory to the FITS the directory you are using.
In addition ISRF_filetype = 0 uses only CMB, useful for testing
10. Notes on versions, bugs etc.
11. Oct 2009 AWS A bug in gen_secondary_positron_source.cc since 2006 resulted in secondary electrons equal to secondary positrons. The positrons were corrected. The bug was fixed by TAP in r343 and documented by AWS in rev 344.
12. r370: MAC OS problem due to galprop.h and Galprop.h being indistiguishable: solved by renaming galprop.h
13. r411: skymap format=0 did now working under OpenMP (previously crashed here).
14. r423: IC_isotropic = 1 (ISRF_filetype = 3) will run on any compiler
IC_anisotropic = 1 (ISRF_filetype = 3) will only run if compiled with OMP, see instructions in this README
15. r429: parameters to control CR electron source distribution. See GALDEF/galdef_54_reltest01 for example and doc/galprop_v54.pdf for description.
r434: defaults to nuclei distribution if parameters absent, so upwards compatible (old galdef files will run as before)
16. r438: AWS: intel/icpc/OMP : secondary electrons and positrons are NULL in some cases when OMP_NUM_THREADS >8. Previously it worked with 16 (number of cores). nucleons and electrons are OK. Reporting this in case anyone observes similar behaviour on other systems. It is not a serious problem at present.
17. r453 AWS: 3D models for B-field (using synchrotron=2): han, han_ran and others produce illegal value at R=0 or x=y=0 and this causes
the electron and positron propagation to give nan, apparent when the nuclei file is written. Work-around is to set r_min = 0.01
or x_min = 0.01 to avoid the exact GC point. A better solution is in preparation.
18. r2160: accurate explicit solution method implemented as described in GALPROP Explanatory Supplement, http://www.mpe.mpg.de/~aws/galprop_explanatory_supplement.pdf
This involved changes to propel.cc, Galdef.cc and Galdef.h.
propel.cc is from r860 (trunk).
New parameters in GALDEF file are solution_method and solution_convergence. Defaults to original method if parameters absent. Add these parameters to GALDEF file
to use the explicit method. Example in galdef_54_reltest09.
19. r2160: works with HEALPix 2.20a; if HEALPix is built with openmp option, then --enable-openmp must be specified in ./configure.
20. r2167: synchrotron updated, corrected Stokes calculation. In 3D use synchrotron=3. Example: galdef_54_reltest10 .
21. r2178: free-free absorption and emission. New parameters free_free_absorption, HII_Te, HII_clumping_factor.
Free-free emission computed whenever synchrotron is computed. Examples: galdef_54_reltest01, galdef_54_reltest10.
22. r2190 convection models with more physical z-variation. See Explanatory Supplement. Example: galdef_54_reltest11.
23. r2214 anisotropic diffusion. new parameters: Diffusion_aniso, D0_zz. See Explanatory Supplement. Examples: galdef_54_reltest13, galdef_54_reltest14.
24. r2242 AWS: 20131023 primary positrons included. Parameter set parallels primary nuclei, electrons: injection spectrum, source distribution, normalization.Included in IC, bremsstrahlung, synchrotron emission. Comes after secondary positrons in output nuclei file (for consistency: primary electrons come after secondary electrons). Examples: galdef_54_reltest15 (2D), galdef_54_reltest16(3D) (NB these examples are not yet based on realistic PAMELA-type excess).
25. r2261 AWS: 20131028 new parameter for boundary conditions: spatial_bound_conds. Adds free escape at boundary option. See Explanatory Supplement for details. Sample cases: galdef_54_reltest17,18 (2D, 3D).
26 r2288 AWS: 20131104 new hadronic gamma-ray models. Control with galdef parameter pi0_decay. To list models, do "grep hadronic_model source/gen_pi0_decay_emiss.cc". See Explanatory Supplement for more details.
27 r2313 AWS: 20131107 bug in CR normalization from 2006 corrected. Changes at % level expected. Check range of normalization Ekin against Ekin_min, Ekin_max for valid interpolation (applies to nuclei, electrons, positrons), to avoid invalid results if out of range.
28 AWS20131108 r2318 source distribution: source_parameters_0 =zscale defaults to 0.2 but older galdef files set it to 0 causing invalid results. This is now checked, if 0 issues an error message an exits. It is recommended to set this parameter explicitly.
29. AWS20131111 r2331 third break in injection spectra for primary nuclei, electrons, positrons. Parameters nuc_rigid_br2, nuc_g_3 etc. Defaults to very high rigidity, so upwards compatible if undefined. nuclei: low-energy break parameters nuc_rigid_br0, nuc_g_0 added consistent with electrons, positrons. Defaults to very low value so upwards compatible if undefined. Exammples of new parameters in galdef_54_reltest17.
30. AWS20131114 r2339 AWS: Bremsstrahlung c++ version used, in gen_bremss_emiss.cc from trunk. Activate with galdef parameter bremss = 2 (instead of usual 1). Still under test but agrees mostly with fortran version (it is just a translation).
31. AWS20131202 configure for HEALPix 3.11. Still works for 2.20a.
32. AWS20131231 r2396: HEALPix skymaps: energies in columns instead of vectors per pixel. Compatible with Aladin and GALPLOT. If the vector format is required, replace SkymapFitsio.cc with the original version provided in SkymapFitsio.cc_vector.
33. AWS20131231 noted problem with g++ (4.3.4) and openmp: secondary electrons and positrons are NaN (hence synchrotron, IC, bremsstrahlung skymaps also affected). OK with intel icpc. To be investigated.
34. AWS20140107 GALDEF/galdef_54_reltest19: example paired with GALPLOT plot package: use GALDEF/galplotdef_54_reltest19 provided.
Also GALDEF/galdef_54_reltest20: primary positrons, no secondary positrons (instead of just secondary positrons in reltest19); example paired with GALPLOT plot package: use GALDEF/galplotdef_54_reltest20 provided.
35. AWS20140126 r2423 added FITS directory with complete reference output for run galdef_54_reltest02. Use to verify installation by comparison with local run. Put corrected revision number in screen ouput (set to 54.1.2423), controlled by configure.ac.
36. AWS20140418 r2439 updated Explanatory Supplement in response to user comments. Very minor debugs, recompilation not essential.
37. AWS20140514 r2481 added secondary deuterium production via pp->d fusion. See Explanatory Supplement Section 5.5.2 for details.
38. AWS20140605 r2504 implemented Jansson and Farrar 2012 B-field model. Examples in galdef_54_reltest21, galdef_54_reltest22. See Explanatory Supplement for details under parameter B_field_name.
39. AWS20140711 galprop may hang in gen_secondary_positron_source.cc: this happened with g++, openmp on a RHEL system at SLAC. The solution was:
setenv OMP_NUM_THREADS 1
to avoid parallel processing. Another solution would be not to configure with openmp. Reason unknown. The problem has not yet been observed elsewhere to our knowledge.
AWS20150630 r2765 solved by removing openmp pragma in gen_secondary_positron_source.cc. This is anyway superfluous. Now runs in parallel on RHEL.
40. AWS20140820 pi0_decay = 11, 12 gives incorrect (very large) results in a narrow range around 1 GeV for Ekin_max = 1e9 MeV. The problem is caught for nan or inf, but not for simply incorrect values. Hence use Ekin_max = 1e8 or less - this is no restriction since GALPROP is anyway not valid above 100 TeV.
41. AWS20150417 r2754 added hadronic energy losses for nuclei. New GALDEF parameters: hadronic_losses, hadronic_loss_model. See Explanatory Supplement for details. Sample parameter file: galdef_54_reltest23.
documented by AWS, updated 20090612: added ccfits to ./configure, for r248, MPE example
documented by AWS, updated 20090721: added CLHEP to ./configure, for r273, MPE example
documented by AWS, updated 20090803: added description of 2 test cases
documented by AWS, updated 20090805: added new configure and runtime options for r293
documented by AWS, updated 20090826 for r304 new installation procedure
documented by AWS, updated 20090902 for r315, new commandline options, message about svn server.
documented by AWS, updated 20090921 for r324 output FITS is controlled by --with-fitsdate if not on command line. CXXFLAGS obsolete. Warning about ISRF_filetype.
documented by AWS, updated 20090923 for r325 options -p , -h
documented by AWS, updated 20090929 for r329 option --enable-debug, new ISRF format instructions
documented by AWS, updated 20091012 for r344 fixed bug in gen_secondary_positron_source.cc
documented by AWS, updated 20100118 for r368 added URLs for external libraries, --with-healpix=<path for healpix>/src/cxx/generic_gcc
documented by AWS, updated 20100123 for r371: described ISRF file behaviour. MAC OS fix.
documented by AWS, updated 20100124 for r372: compiler and OpenMP information
documented by AWS, updated 20100427 bugfix for skymap_format=0 OpenMP
documented by AWS, updated 20100505 new ISRFs for anisotropic IC described
documented by AWS, updated 20100507 new CR electron source parameters described
documented by AWS, updated 20100621 added howto redirect to file. Noted bug in 3D B-field.
documented by AWS, updated 20131001 added information on explicit method. Note on HEALPix 2.20a
documented by AWS, updated 20131004 synchrotron, free-free updates
documented by AWS, updated 20131008 new convection models
documented by AWS, updated 20131017 anisotropic diffusion
documented by AWS, updated 20131023 primary positrons
documented by AWS, updated 20131028 boundary conditions parameter
documented by AWS, updated 20131031 note on static, runtime libraries
documented by AWS, updated 20131104 new hadronic models
documented by AWS, updated 20131107 2 bug fixes
documented by AWS, updated 20131108 1 bug fix
documented by AWS, updated 20131111 3rd break in injection spectra, low energy break for nucleons
documented by AWS, updated 20131114 c++ bremsstrahlung option
documented by AWS, updated 20131202 HEALPix 3.11 support
documented by AWS, updated 20131204 ./configure option FC
documented by AWS, updated 20131231 HEALPix output changed; note on g++ openmp bug.
documented by AWS, updated 20140107 reltest19, 20 described
documented by AWS, updated 20140126 reference output in FITS
documented by AWS, updated 20140418
documented by AWS, updated 20140514: r2481
documented by AWS, updated 20140605: r2504
documented by AWS, updated 20140711: openmp bug
documented by AWS, updated 20140820 pi0-decay bug
documented by AWS, updated 20150227 r2715 minor bug fixes including bugzilla #88
documented by AWS, updated 20150417 r2754
documented by AWS, updated 20150630 r2765 see 39.
