Accelerating an r-process calculation

In my last post, I ran a full r-process calculation. The calculation, which took 15 hours on my laptop, used the default "arrow" matrix solver in wn_matrix. In this post I will show how I used a Krylov-space iterative solver to accelerate the r-process calculation.

SPARSKIT is a basic tool-kit for sparse matrix calculations and is written and maintained by Yousef Saad. Our code module wn_sparse_solve interfaces our basic matrix routines in wn_matrix with SPARSKIT.

To use SPARSKIT2, the latest version, in the r-process calculation, I first set the environment variable NNT_USE_SPARSKIT2 and recompiled the network code. The network code can either be the base example codes in examples/network, or, for my present purposes, the one with my user-defined trajectory. I typed:

cd nucnet-tools-code/my_examples/network

I set the environment variable

export NNT_USE_SPARSKIT2=1

and cleaned and remade the code

make clean

make all_network

This downloaded wn_sparse_solve and SPARSKIT2 and rebuilt the network code. SPARSKIT2 is written in Fortran, so the default compilation will use gfortran. gfortran should automatically be part of the gcc compiler collection, so you should have no problem with the compilation, but, if you do, you might update gcc.

Once you have successfully compiled the code, you can run it as before. I used the following ../../data_pub/zone.xml file:

<zone_data>
<zone>
  <optional_properties>
    <property name="tend">1.e6</property>
    <property name="tau_0">0.035</property>
    <property name="tau_1">1.</property>
    <property name="munuekT">-inf</property>
    <property name="t9_0">10.</property>
    <property name="steps">20</property>
    <property name="rho_0">1.4985e6</property>
    <property name="rho_1">1.5e3</property>
    <property name="iterative solver method">gmres</property>
    <property name="t9 for iterative solver">2.</property>
  </optional_properties>
  <mass_fractions>
    <nuclide name="n">
      <z>0</z>
      <a>1</a>
      <x>0.67</x>
    </nuclide>
    <nuclide name="h1">
      <z>1</z>
      <a>1</a>
      <x>0.33</x>
    </nuclide>
  </mass_fractions>
</zone>
  </zone_data>

Notice the addition of the two properties iterative solver method and t9 for iterative solver. The former tells the code to use the iterative solver method gmres. I've had good luck with that solver, but you can try other solvers such as bicgstab. See wn_sparse_solve for more choices. The latter property tells the code when to start using the iterative solver instead of the default solver. In the above, I set the property so that the code used the gmres solver for temperatures below T9 = 2. Experience has shown that above this temperature, the iterative solver has trouble converging, at least for the default preconditioner.

With these preliminaries completed, I ran the code by typing:

./run_single_zone ../../data_pub/my_net.xml ../../data_pub/zone.xml my_output.xml "[z <= 90]"

The execution time on my laptop dropped to 3.5 hours. This is better than a factor of four increase in execution speed. In my next posts, I will explore the reason for this speed up.

Posted by Bradley S. Meyer 2013-12-08
  • Anonymous - 2015-03-02

    Dear Brad,

    After I set the environment value "export NNT_USE_SPARSKIT2=1"
    I type

    make clean_all

    make all_network

    I get the following errors
    "In file included from ../../user/matrix_solver.cpp:21:
    ../../user/matrix_solver.h:61: error: wrong number of template arguments (1, should be 2)
    /usr/include/boost/detail/container_fwd.hpp:80: error: provided for 'template<class T,="" class="" Allocator=""> struct std::vector'
    ../../user/matrix_solver.cpp:387: error: wrong number of template arguments (1, should be 2)
    /usr/include/boost/detail/container_fwd.hpp:80: error: provided for 'template<class T,="" class="" Allocator=""> struct std::vector'
    ../../user/matrix_solver.cpp: In function 'gsl_vector user::phisolveparallel(WnSparseSolve__Phi, int&, const gsl_vector, const gsl_vector, double)':
    ../../user/matrix_solver.cpp:404: error: request for member 'size' in 'matrices', which is of non-class type 'int'
    ../../user/matrix_solver.cpp:424: error: invalid types 'int[int]' for array subscript
    ../../user/matrix_solver.cpp:437: error: wrong number of template arguments (1, should be 2)
    /usr/include/boost/detail/container_fwd.hpp:80: error: provided for 'template<class T,="" class="" Allocator=""> struct std::vector'
    ../../user/matrix_solver.cpp:437: error: invalid type in declaration before '(' token
    ../../user/matrix_solver.cpp:437: error: request for member 'size' in 'matrices', which is of non-class type 'int'
    ../../user/matrix_solver.cpp:490: error: request for member 'size' in 'matrices', which is of non-class type 'int'
    ../../user/matrix_solver.cpp:492: error: invalid types 'int[size_t]' for array subscript
    ../../user/matrix_solver.cpp:494: error: invalid types 'int[size_t]' for array subscript
    ../../user/matrix_solver.cpp:490: error: invalid controlling predicate
    ../../user/matrix_solver.cpp:501: error: request for member 'size' in 'matrices', which is of non-class type 'int'
    ../../user/matrix_solver.cpp:511: error: invalid types 'int[size_t]' for array subscript
    ../../user/matrix_solver.cpp:512: error: invalid types 'int[size_t]' for array subscript
    ../../user/matrix_solver.cpp:514: error: invalid types 'int[size_t]' for array subscript
    ../../user/matrix_solver.cpp:522: error: invalid types 'int[size_t]' for array subscript
    ../../user/matrix_solver.cpp:522: error: invalid types 'int[size_t]' for array subscript
    ../../user/matrix_solver.cpp:523: error: invalid types 'int[size_t]' for array subscript
    ../../user/matrix_solver.cpp:532: error: invalid types 'int[int]' for array subscript
    ../../user/matrix_solver.cpp:534: error: invalid types 'int[int]' for array subscript
    make: *** [../../obj/matrix_solver.o] Error 1
    "

    Thank you very much!!!

     
    • Bradley S. Meyer

      Thanks for your post. I get the same problem if I compile against an old version of Boost but not against an up-to-date version. The default version of Boost on one of our computers is 1.41.0, which I can find by typing

      cd nucnet-tools-code/examples/misc

      make test_versions

      ./test_versions

      When I compile against the old version, I get the above errors. When I compile against a more recent version or the Boost development branch, everything works.

      Your immediate solution might be to upgrade Boost on your computer or, if you don't have superuser privileges, to install it via git as in the above post. Then recompile against that. I'll nevertheless take a look at the code to see if the error is wholly on the old version of Boost's side. Thanks again, and best wishes.

       
      • Anonymous - 2015-03-03

        Dear Brad,

        Thank you very much!!! Now it works very well!

        Sorry for a lot of questions.

        I met another question!

        When I type

        export NNT_USE_SPARSKIT2=1

        make clean

        make run_single_zone

        It works very well!!! It means that I can run the r-process calculation.

        However when I type

        "make all_network" instead of "make run_single_zone"

        I got the error in the following

        run_entropy.cpp:30:28: error: gsl/gsl_odeiv2.h: No such file or directory
        run_entropy.cpp: In function 'int main(int, char)':
        run_entropy.cpp:583: error: expected initializer before '*' token
        run_entropy.cpp:585: error: 'gsl_odeiv2_step' was not declared in this scope
        run_entropy.cpp:585: error: 'p_step' was not declared in this scope
        run_entropy.cpp:585: error: 'p_T' was not declared in this scope
        run_entropy.cpp:585: error: 'gsl_odeiv2_step_alloc' was not declared in this scope
        run_entropy.cpp:587: error: 'gsl_odeiv2_control' was not declared in this scope
        run_entropy.cpp:587: error: 'p_c' was not declared in this scope
        run_entropy.cpp:587: error: 'gsl_odeiv2_control_y_new' was not declared in this scope
        run_entropy.cpp:589: error: 'gsl_odeiv2_evolve' was not declared in this scope
        run_entropy.cpp:589: error: 'p_e' was not declared in this scope
        run_entropy.cpp:589: error: 'gsl_odeiv2_evolve_alloc' was not declared in this scope
        run_entropy.cpp:591: error: 'gsl_odeiv2_system' was not declared in this scope
        run_entropy.cpp:591: error: expected ';' before 'sys'
        run_entropy.cpp:629: error: 'sys' was not declared in this scope
        run_entropy.cpp:634: error: 'gsl_odeiv2_evolve_apply' was not declared in this scope
        run_entropy.cpp:831: error: 'gsl_odeiv2_evolve_free' was not declared in this scope
        run_entropy.cpp:832: error: 'gsl_odeiv2_control_free' was not declared in this scope
        run_entropy.cpp:833: error: 'gsl_odeiv2_step_free' was not declared in this scope
        make:
        * [run_entropy] Error 1

        It looks like GSL version is old???

        Because I don't have superuser privileges. Is there a good way to solve it?

        Sorry again for a lot of questions

        Thank you very much!!!

        Best Wishes!

         
        • Bradley S. Meyer

          The odeiv2 package is new in gsl 1.15, so if your gsl is earlier than 1.15, you'll get this error. Try changing every occurrence of odeiv2 to odeiv in the main code run_entropy.cpp and recompiling. That should work. Best wishes.

           
  • Comment has been marked as spam. 
    Undo

    You can see all pending comments posted by this user  here

    Anonymous - 2016-04-21

    Dear Dr brad:
    I ran the code just like you metioned in this post and I got a correct result.But,when I change the mass excess in my_net.xml and renamed it as my_net_WS.xml,I got somethig like this:
    time(s) = 1000000 t9 = 1.00E-04 rho(g/cc) = 1.50E-09
    0 0.00E+00
    1 -4.85E+98
    2 7.76E+111
    3 2.46E+118
    4 -4.13E+89
    5 0.00E+00
    6 0.00E+00
    7 4.36E+117
    8 -7.58E+158
    9 2.65E+205
    10 -1.86E+126
    11 1.79E+140
    12 1.98E+144
    13 1.52E+142
    14 2.43E+147
    15 2.84E+140
    16 -3.88E+152
    17 1.21E+202
    18 -6.24E+144
    19 3.14E+127
    20 -8.21E+129
    21 3.27E+197
    22 2.09E+199
    23 2.64E+142
    24 -7.68E+137
    25 -7.23E+134
    26 -1.67E+185
    27 -2.67E+196
    28 -1.41E+189
    29 6.68E+130
    30 7.14E+131
    31 2.53E+196
    32 1.18E+171
    33 -2.59E+108
    34 -5.80E+122
    35 1.36E+133
    36 -6.94E+147
    37 2.25E+133
    38 -4.38E+177
    39 3.92E+113
    40 -9.35E+114
    41 -8.65E+177
    42 6.26E+133
    43 9.09E+112
    44 3.49E+177
    45 -5.06E+108
    46 1.61E+124
    47 -5.70E+142
    48 -9.24E+145
    49 4.66E+132
    50 -5.76E+117
    51 -5.28E+109
    52 -3.46E+173
    53 -1.39E+76
    54 2.32E+131
    55 -8.29E+75
    56 3.57E+130
    57 2.92E+48
    58 4.37E+67
    59 1.36E+131
    60 1.54E+113
    61 -1.47E+51
    62 -6.23E+48
    63 -5.83E+50
    64 -1.85E+100
    65 -8.51E+37
    66 2.77E+105
    67 -2.15E+49
    68 2.30E+113
    69 1.42E+47
    70 -5.62E+114
    71 -3.59E-14
    72 -1.89E-14
    73 -2.62E+52
    74 -3.47E+52
    75 -1.35E+52
    76 -4.40E+12
    77 -4.63E+77
    78 -3.69E+53
    79 -1.93E+25
    80 1.53E+113
    81 -8.20E+53
    82 2.12E+31
    83 -7.79E+65
    84 -4.22E+65
    85 -3.72E+147
    86 -1.23E+88
    87 -1.39E+89
    88 -3.06E+22
    89 -1.29E+88
    90 -7.55E+87
    91 -1.85E+88
    92 -9.33E+152
    93 -1.37E+88
    94 -5.99E+88
    95 -1.13E+87
    96 -2.75E+84
    97 -1.91E+87
    98 -2.55E+87
    99 -1.27E+86
    100 -1.39E+86
    101 -8.88E+85
    102 -4.30E+86
    103 -6.37E+84
    104 -5.09E+18
    105 -7.68E+17
    106 -6.97E+10
    107 -1.15E+11
    108 1.33E+27
    109 -3.53E-12
    110 -5.48E+18
    111 -3.90E+14
    112 -2.98E+14
    113 -1.09E+82
    114 -3.83E+82
    115 -4.10E+12
    116 -4.57E+16
    117 -1.15E+16
    118 -1.34E+80
    119 -6.18E+14
    120 -1.74E+16
    121 -3.57E+00
    122 -3.25E+00
    123 5.33E+69
    124 -9.08E+04
    125 -3.44E+03
    126 -1.62E+05
    127 -5.63E+03
    128 -1.65E+05
    129 -1.09E+04
    130 -4.18E+05
    131 -3.00E+27
    132 -3.23E+92
    133 -2.31E+55
    134 -2.82E+69
    135 -3.99E+55
    136 4.36E+53
    137 -1.04E-11
    138 -8.38E+38
    139 -3.39E+102
    140 1.79E+54
    141 -3.71E+52
    142 -5.42E+53
    143 3.40E+54
    144 1.49E+53
    145 -1.11E+54
    146 -3.23E+53
    147 3.74E+54
    148 -1.55E+54
    149 -1.16E+54
    150 2.15E+54
    151 -1.40E+54
    152 -1.55E+54
    153 -5.15E+80
    154 -5.68E+54
    155 -2.77E+55
    156 -9.27E+35
    157 -1.01E+33
    158 -4.38E+32
    159 -9.27E+32
    160 -1.47E+99
    161 -1.66E+98
    162 -1.09E+55
    163 2.62E+55
    164 -2.76E+54
    165 -1.71E+55
    166 1.44E+55
    167 -1.13E+55
    168 -6.77E+54
    169 1.37E+55
    170 -8.21E+54
    171 -4.01E+54
    172 8.33E+54
    173 -5.05E+54
    174 -3.23E+54
    175 3.62E+54
    176 -2.29E+54
    177 -2.30E+54
    178 6.34E+54
    179 -5.45E+54
    180 -1.56E+54
    181 1.21E+55
    182 -4.03E+54
    183 -1.06E+54
    184 -8.04E+53
    185 -1.60E+54
    186 5.97E+54
    187 -2.48E+52
    188 1.08E+70
    189 -4.81E+53
    190 1.51E+95
    191 7.82E+54
    192 9.96E+87
    193 1.39E+96
    194 5.62E+55
    195 -1.02E+05
    196 -1.42E+55
    197 2.91E+55
    198 3.13E+98
    199 -1.29E+55
    200 -3.30E+54
    201 1.74E+99
    202 -6.45E+52
    203 5.70E+59
    204 -9.42E+52
    205 2.16E+33
    206 5.87E+50
    207 8.55E+96
    208 1.77E+69
    209 1.18E+05
    210 3.03E+25
    211 4.60E+72
    212 2.11E+51
    213 2.53E+88
    214 6.44E+23
    215 1.52E+52
    216 3.91E+88
    217 7.23E+88
    218 -1.41E-11
    219 3.60E+23
    220 3.18E+81
    221 -4.40E+52
    222 -3.48E+51
    223 1.41E+53
    224 9.11E+88
    225 5.58E+52
    226 4.57E+88
    227 5.35E+53
    228 -5.25E+52
    229 2.39E+30
    230 8.62E+88
    231 2.35E+96
    232 -6.00E+52
    233 -4.34E+52
    234 3.54E+53
    235 7.89E+95
    236 -1.05E+53
    237 1.22E+95
    238 0.00E+00
    239 0.00E+00
    240 0.00E+00
    241 0.00E+00
    242 0.00E+00
    243 0.00E+00
    244 0.00E+00
    245 0.00E+00
    246 0.00E+00
    247 0.00E+00
    248 0.00E+00
    249 0.00E+00
    250 0.00E+00
    251 0.00E+00
    252 0.00E+00
    253 0.00E+00
    254 0.00E+00
    255 -4.05E-16
    256 6.09E-17
    257 1.13E-16
    258 3.75E-15
    259 2.30E-15
    260 6.64E-15
    261 5.35E-16
    262 -3.02E-13
    263 0.00E+00
    264 0.00E+00
    265 0.00E+00
    266 0.00E+00
    267 0.00E+00
    268 0.00E+00
    269 0.00E+00
    270 0.00E+00
    271 0.00E+00
    272 0.00E+00
    273 -6.66E-15
    274 4.86E+95
    275 0.00E+00
    276 0.00E+00
    277 0.00E+00
    278 0.00E+00
    279 0.00E+00
    280 0.00E+00
    281 0.00E+00
    282 0.00E+00
    283 0.00E+00
    284 0.00E+00
    285 0.00E+00
    286 0.00E+00
    287 0.00E+00
    288 0.00E+00
    289 0.00E+00
    290 0.00E+00
    291 0.00E+00
    292 0.00E+00
    293 0.00E+00
    As you see,the abundances are too large,I want to know what's the problem with my caculation.
    best wishes!

     
    • Bradley S. Meyer

      I've moved this to the dicussion board.

       


Anonymous

Cancel  Add attachments





Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks