Re: [sparta-users] [EXTERNAL] read_restart with very large files

[Moore, S. <st...@sa...>]

Hi Cosimo,

The global mem/limit is not implemented for the single file read restart yet. You need to use multiple file readers for global mem/limit to work with the restart file. I will add an error message to the code.

 If a "%" character appears in the filename, then one file is written by each processor and the "%" character is replaced with the processor ID from 0 to P-1. An additional file with the "%" replaced by "base" is also written, which contains global information.

Stan
________________________________
From: Cosimo Livi via sparta-users <spa...@li...>
Sent: Friday, February 4, 2022 6:03 AM
To: spa...@li... <spa...@li...>
Subject: [EXTERNAL] [sparta-users] read_restart with very large files


Dear community,



I am working with very large simulations (20e9 particles), and due to the large simulation time I wanted to save restart files to split the simulation in smaller parts.



The problem is that my restart files have a size of 1.9 TB, and when I try to read them I get the following output and error:

“

SPARTA (7 Jan 2022)

Reading restart file ...

  restart file = 7 Jan 2022, SPARTA = 7 Jan 2022

  orthogonal box = (0 0 0) to (0.184 0.102 0.03)

ERROR on proc 0: Restart file read buffer too large, use global mem/limit (../read_restart.cpp:198)

”



Following the error suggestion, I have tried to use the global mem/limit instruction before reading the restart, but the output does not change and I still get the same error.

The relevant lines in the input file are:

“

global                   mem/limit 1800

read_restart        after_warmup.restart

“



I am running SPARTA (7 Jan 2022) release compiled with BIGBIG flag



There is a way to restart such large simulations?

Thank you in advance for you invaluable help and support,



Cosimo Livi

-- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. Neither the sender nor the company/group of companies he or she represents shall be liable for the proper and complete transmission of the information contained in this communication, or for any delay in its receipt. -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. Unless explicitly stated otherwise in the body of this communication or the attachment thereto (if any), the information is provided on an AS-IS basis without any express or implied warranties or liabilities. To the extent you are relying on this information, you are doing so at your own risk. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. Neither the sender nor the company/group of companies he or she represents shall be liable for the proper and complete transmission of the information contained in this communication, or for any delay in its receipt.