Reducing disk writing load of redundant data with single-zone runs?

[Leonard Nielsen]

Dear professor Meyer,

I am running a customized r-process calculation; the code is based off run_single_zone with a modified density trajectory and a constant-entropy temperature evolution. I have noticed however when running a large number of parallell processes (for the purpose of a Monte Carlo calculation) that the disk writing to zone output size ratio is very poor. It looks like the code requires on the order of 50 MB writing per step per calculation, even though an 1800-step zone file is only ~300 MB. My guess is that it re-writes the nuclear network file into the output at every step or something along those lines since the network file is about the right size; this acts as a bottleneck. My present solution is simply to increase the step size by an order of magnitude or so, but I may wish to look at statistics of neutron flows or certain reaction channels at some point in the future, so I wonder if there is a way to eliminate what appears to be a redundant writing process.

The offending function must be one of the following, under "Print out abundances" in run_single_zone.cpp:

if(
       (
         i_step % zone.getProperty<size_t>( nnt::s_STEPS ) == 0 ||
         d_t >= zone.getProperty<double>( nnt::s_TEND )
       )
    )
    {
      Libnucnet__relabelZone(
        p_my_nucnet,
        zone.getNucnetZone(),
        ( boost::lexical_cast<std::string>( ++k ) ).c_str(),
        NULL,
        NULL
      );
      nnt::print_zone_abundances( zone );
      nnt::write_xml( p_my_output, zone.getNucnetZone() );
      Libnucnet__writeToXmlFile(
        p_my_output,
        zone.getProperty<std::string>( S_OUTPUT_FILE ).c_str()
      );
    }

and if I am correct it is write_xml, and in that code, under nnt/auxilliary, possibly this?

 Libnucnet__Zone * p_new_zone;
  gsl_vector * p_vector;
  char s_zone[i_BUF_SIZE];

or this

  p_new_zone =
    Libnucnet__Zone__new(
      Libnucnet__getNet( p_zone_data ),
      s_zone,
      NULL,
      NULL
    );

where I don't seem to get further as I don't seem to have the source code for the Libnucnet functions. So, is it possible to streamline the built-in functions to eliminate the redundant writing, or will I have to work around this limitation?

[Bradley S. Meyer]

Thanks for your question. First, the libnucnet source code is available at libnucnet in case you want to poke around there. Second, what's going on in write_xml is that the current zone (time step data) is being copied to the output. It first gets relabeled to have an index one more than the previous output. Then that zone is copied and added to the output. These should all be fast and low memory. I'm guessing where you are having the slowdown is in writeToXmlFile(). That does indeed write the reaction network, etc. It is probably unnecessary for your purposes. That is why in my example I have

  Libnucnet__relabelZone(
    p_my_nucnet,
    zone.getNucnetZone(),
    ( boost::lexical_cast<std::string>( ++k ) ).c_str(),
    NULL,
    NULL
  );
  nnt::print_zone_abundances( zone );
  nnt::write_xml( p_my_output, zone.getNucnetZone() );
  if( B_OUTPUT_EVERY_TIME_DUMP )
  {
    Libnucnet__writeToXmlFile(
      p_my_output,
      zone.getProperty<std::string>( S_OUTPUT_FILE ).c_str()
    );
  }

The boolean B_OUTPUT_EVERY_TIME_DUMP (default false) checks whether you want to write out every time dump or not. You might simply want to eliminate that call--just make sure you have the output at the end of the calculation--that is, you need the

Libnucnet__writeToXmlFile(
  p_my_output,
  zone.getProperty<std::string>( S_OUTPUT_FILE ).c_str()
);

//============================================================================
// Clean up and exit.
//============================================================================

Libnucnet__free( p_my_output );

at the end to dump out the final XML with all the data. That will still copy the network data and be the big disk hog, but only once at the end of the calculation.

If you still have trouble, you can switch over to outputing HDF5, but, as you say, with the XML, you can make the flow diagrams, etc.

Executive summary:

I suggest you simply delete

  Libnucnet__writeToXmlFile(
    p_my_output,
    zone.getProperty<std::string>( S_OUTPUT_FILE ).c_str()
  );

in the first block in your question, or at least. guard it with some kind of if statement that is generally false. Let me know how it goes. Best wishes.

[Leonard Nielsen]

Thank you! I could have sworn I had tried that and found it did not work - but I see now that I got it backwards and deleted the "if" statement instead of the writing function while messing around with the section...I suppose this also stops the Terminal output, but I only need that for debugging anyway.

[Bradley S. Meyer]

I hope all this solves the disk writing issue. Let me know.

The terminal output is from the

  nnt::print_zone_abundances( zone );

line. You could delete that as well or put it inside an if...then debugging block.

Also FYI, I have new versions of the network code coming that simplifies a lot of these issues. I'll announce it on the blog when ready (probably in a month or two). Best wishes.

[Leonard Nielsen]

Thanks! That's great to know! I'm currently working on a Master's Thesis largely regarding the significance of GW170817 measurements for the weak r-process and the use of recent unstable beam facilities to perform experiments in relation to this. NucNet Tools is by far the most easy-to-use, customizable and well-documented code I have found. I appreciate your work very much!