Recent changes to testing

testing modified by T R Jaffe

T R Jaffe — Thu, 16 Mar 2017 19:07:22 -0000

--- v2
+++ v3
@@ -2,22 +2,22 @@

 For a simple test, get the basic parameter file, [test_params.txt](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/test_params.txt/download).  For testing the version linked to Galprop, then try [test2_params.txt](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/test2_params.txt/download).  For the latter, you will need the [GALDEF](https://sourceforge.net/projects/hammurabicode/files/suplementary/GALDEF/) and  [negrid_n400.bin](https://sourceforge.net/projects/hammurabicode/files/suplementary/negrid_n400.bin/download) files.  

-For far more examples of different ways to run the code, there is a set of tar files containing a set of "unit tests".  This consists of many different runs of hammurabi with different parameters designed to test various functions of the code, various test cases and contexts.  They are not meant to be physically realistic (many are completely unphysical) but simply to control for changes in the results.  This was also originally not meant to be public, so it's not terribly user friendly.  Suggestions welcome.  
+For far more examples of different ways to run the code, there is a set of "unit tests".  This consists of many different runs of hammurabi with different parameters designed to test various functions of the code, various test cases and contexts.  They are not meant to be physically realistic (many are completely unphysical) but simply to control for changes in the results.  This was also originally not meant to be public, so it's not terribly user friendly.  Suggestions welcome.  

-The file [hammurabi_unit_test_public.tgz](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/hammurabi_unit_test_public.tgz/download) contains a script, test.csh, which calls hammurabi different ways with a brief description of the point of the test.  You can simply browse this and then look at the included parameter files for each test to get an idea how to run the code different ways.  
+The test scripts are not part of the release, so you'll need to check out the code from the svn repository, which will include the unit_test subdirectory.  Then in that subdirectory, untar the inputs found in the file [hammurabi_unit_test_inputs.tgz](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/hammurabi_unit_test_inputs.tgz/download).  You can simply browse the test script (test.csh) to see the cases run, and/or then look at the included parameter files for each test to get an idea how to run the code different ways.  

 Or you could try to run the whole thing.  Dependencies:

 - all hammurabi targets must be built, including the versions linked with GALPROP and NE2001 codes as well as the debug version ("make all");  

-- IDL, in order to create visualizations of the results;
+- IDL, python, healpy, hampy (the python tools of hammurabi in the svn), etc. in order to create visualizations of the results;

 - the HEALPix Fortran and IDL functions, so that the IDL wrappers for tools like anafast work;  

 - gs (ghostscript), in order to convert PS files written by IDL into PNGs for visualizing on a simple web page (on my system, I have issues writing the images directly to PNG).

-1)  Download the tar file for the tests, i.e. the  [hammurabi_unit_test_public.tgz](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/hammurabi_unit_test_public.tgz/download),   and expand it in the directory where hammurabi is built.  It consists of a unit_test subdirectory that contains the parameter files and scripts.  Then under unit_test, expand the  [hammurabi_unit_test_public_inputs.tgz](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/hammurabi_unit_test_public_inputs.tgz/download).  
+1)  Under the unit_test subdirectory, expand the  [hammurabi_unit_test_public_inputs.tgz](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/hammurabi_unit_test_public_inputs.tgz/download).  

 2) Then from the main hammurabi directory, simply 

@@ -34,7 +34,7 @@
 and it will skip to the plotting step rather than wiping out the previous results (!) and starting over.  Then you can work your way through problems in the plotting.  

-3) If it runs successfully, you can compare the outputs to my previous runs.  Get the appropriate "ref" set (ref or ref.mini) and untar it under unit_test.  Then start by comparing two web pages in your browser, each of which shows a few plots for each test case, e.g.:
+3) If it runs successfully, you can compare the outputs to my previous runs.  Get the tar file containing the reference data from  [hammurabi_unit_test_public_ref.tgz](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/hammurabi_unit_test_ref.tgz/download) and untar it under unit_test.  Then start by comparing two web pages in your browser, each of which shows a few plots for each test case, e.g.:

 unit_test/ref/views/plots.html

testing modified by T R Jaffe

T R Jaffe — Fri, 03 Jun 2016 11:03:32 -0000

--- v1
+++ v2
@@ -1,8 +1,12 @@
 Testing hammurabi:

-On the Sourceforge site can be found a tar file containing a set of "unit tests".  This consists of many different runs of hammurabi with different parameters designed to test various functions of the code, various test cases and contexts.  They are not meant to be physically realistic (many are completely unphysical) but simply to control for changes in the results.  This was also originally not meant to be public, so it's not terribly user friendly.  Suggestions welcome.  
+For a simple test, get the basic parameter file, [test_params.txt](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/test_params.txt/download).  For testing the version linked to Galprop, then try [test2_params.txt](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/test2_params.txt/download).  For the latter, you will need the [GALDEF](https://sourceforge.net/projects/hammurabicode/files/suplementary/GALDEF/) and  [negrid_n400.bin](https://sourceforge.net/projects/hammurabicode/files/suplementary/negrid_n400.bin/download) files.  

-Dependencies:
+For far more examples of different ways to run the code, there is a set of tar files containing a set of "unit tests".  This consists of many different runs of hammurabi with different parameters designed to test various functions of the code, various test cases and contexts.  They are not meant to be physically realistic (many are completely unphysical) but simply to control for changes in the results.  This was also originally not meant to be public, so it's not terribly user friendly.  Suggestions welcome.  
+
+The file [hammurabi_unit_test_public.tgz](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/hammurabi_unit_test_public.tgz/download) contains a script, test.csh, which calls hammurabi different ways with a brief description of the point of the test.  You can simply browse this and then look at the included parameter files for each test to get an idea how to run the code different ways.  
+
+Or you could try to run the whole thing.  Dependencies:

 - all hammurabi targets must be built, including the versions linked with GALPROP and NE2001 codes as well as the debug version ("make all");  

@@ -13,13 +17,15 @@
 - gs (ghostscript), in order to convert PS files written by IDL into PNGs for visualizing on a simple web page (on my system, I have issues writing the images directly to PNG).

-1)  Download the tar file for the tests and expand it in the directory where hammurabi is built.  It consists of a unit_test subdirectory that contains the inputs, parameter files, scripts, and reference outputs for comparison.  
+1)  Download the tar file for the tests, i.e. the  [hammurabi_unit_test_public.tgz](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/hammurabi_unit_test_public.tgz/download),   and expand it in the directory where hammurabi is built.  It consists of a unit_test subdirectory that contains the parameter files and scripts.  Then under unit_test, expand the  [hammurabi_unit_test_public_inputs.tgz](https://sourceforge.net/projects/hammurabicode/files/suplementary/testing/hammurabi_unit_test_public_inputs.tgz/download).  

 2) Then from the main hammurabi directory, simply 

 >  make test

-Depending on the system and whether the code is built with openMP, this can take a long time (hours).  Before it starts running hammurabi, it checks your available memory.  If you have 3GB free memory, it runs the normal test, which includes some GRF simulations that require that much.  If not, it runs a mini-test, the largest of which requires about 1GB.  (Note that on OSX, things in memory are moved around dynamically, so the check may fail but the test would work fine if you tried.  If you want to force it, edit the unit_test/test.csh script.)
+(Alternatively, under unit_test, run simply "./test.csh".)
+
+Depending on the system and whether the code is built with openMP, this can take a long time (hours).  Before it starts running hammurabi, it checks your available memory.  If you have 3GB free memory, it runs the normal test, which includes some GRF simulations that require that much.  If not, it runs a mini-test, the largest of which requires about 1GB.  (Note that the check may fail but the test would work fine if you tried.  If you want to force it, edit the unit_test/test.csh script.)

 This command runs the script in unit_test/test.csh, which you can run manually in several ways.  For example, if the hammurabi tests run but you are having trouble generating the plots, then you can run 

@@ -28,7 +34,7 @@
 and it will skip to the plotting step rather than wiping out the previous results (!) and starting over.  Then you can work your way through problems in the plotting.  

-3) If it runs successfully, you can compare the outputs to my previous runs.  Get the appropriate "ref" set (ref, ref.mini, or ref.big) and untar it under unit_test.  Then start by comparing two web pages in your browser, each of which shows a few plots for each test case, e.g.:
+3) If it runs successfully, you can compare the outputs to my previous runs.  Get the appropriate "ref" set (ref or ref.mini) and untar it under unit_test.  Then start by comparing two web pages in your browser, each of which shows a few plots for each test case, e.g.:

 unit_test/ref/views/plots.html

testing modified by T R Jaffe

T R Jaffe — Fri, 03 Jun 2016 10:29:23 -0000

Testing hammurabi:

On the Sourceforge site can be found a tar file containing a set of "unit tests". This consists of many different runs of hammurabi with different parameters designed to test various functions of the code, various test cases and contexts. They are not meant to be physically realistic (many are completely unphysical) but simply to control for changes in the results. This was also originally not meant to be public, so it's not terribly user friendly. Suggestions welcome.

Dependencies:

all hammurabi targets must be built, including the versions linked with GALPROP and NE2001 codes as well as the debug version ("make all");
IDL, in order to create visualizations of the results;
the HEALPix Fortran and IDL functions, so that the IDL wrappers for tools like anafast work;
gs (ghostscript), in order to convert PS files written by IDL into PNGs for visualizing on a simple web page (on my system, I have issues writing the images directly to PNG).

1) Download the tar file for the tests and expand it in the directory where hammurabi is built. It consists of a unit_test subdirectory that contains the inputs, parameter files, scripts, and reference outputs for comparison.

2) Then from the main hammurabi directory, simply

make test

Depending on the system and whether the code is built with openMP, this can take a long time (hours). Before it starts running hammurabi, it checks your available memory. If you have 3GB free memory, it runs the normal test, which includes some GRF simulations that require that much. If not, it runs a mini-test, the largest of which requires about 1GB. (Note that on OSX, things in memory are moved around dynamically, so the check may fail but the test would work fine if you tried. If you want to force it, edit the unit_test/test.csh script.)

This command runs the script in unit_test/test.csh, which you can run manually in several ways. For example, if the hammurabi tests run but you are having trouble generating the plots, then you can run

(cd unit_test; ./test.csh plots)

and it will skip to the plotting step rather than wiping out the previous results (!) and starting over. Then you can work your way through problems in the plotting.

3) If it runs successfully, you can compare the outputs to my previous runs. Get the appropriate "ref" set (ref, ref.mini, or ref.big) and untar it under unit_test. Then start by comparing two web pages in your browser, each of which shows a few plots for each test case, e.g.:

unit_test/ref/views/plots.html

versus

unit_test/out/views/plots.html