Re: [Dmtcp-forum] Why does the restarted process load everything into physical memory?

[Gene C. <ge...@cc...>]

You are correct.  In order to have a clean, general solution, upon restart,
we simply copy the data from a .so file into the proper location in memory.
Among other implications, this does break sharing of .so images between
multiple processes.  This is done because the checkpoint image could
even be copied to a new host, and the original .so files (embedded in
the checkpoint image) are also copied to the new host.

We are experimenting with a new option (to be available in the next
release, or now in the svn) whereby one can create an _uncompressed_
checkpoint image, and then on restart, one will mmap it into memory.
In this case, pages are loaded into memory only on demand, which should
reduce the initial memory requirements and allow for faster restarts.
I'd expect that you'd see some benefits on very large checkpoint images,
and also some smaller speedup because you don't need to execute gzip.
And of course, the checkpoint image on disk will be still larger, since
it's uncompressed.

Even in the next release, this will not yet be made available as a general
option, since we want to do further robustness testing.  However, if
you want to experiment with it, then try:
  env CFLAGS=FAST_CKPT_RST_VIA_MMAP ./configure 
						Best wishes,
						- DMTCP Developers

On Mon, May 16, 2011 at 03:10:37PM -0700, Ying Zheng wrote:
> I checked the /proc/PID/smaps file. For every virtual memory area,
> SIZE=RSS=Private_Dirty.
> Does this mean everything has been loaded into physical memory and cannot be
> shared with other process?
> 
> In my understanding, the shared .so files are just mapped into virtual
> memory with read-only or copy-on-write.
> In the original process, only a fraction of mapped pages is really loaded
> into the physical memory. (RSS < Size for most areas)
> 
> But after restarting everything is in RSS and every page is dirty.
> 
> According to the code in mtcp_restart_nolibc.c, the .so files are opened
> read only and is mmap to the virtual address
> 
> if (area.name[0] == '/') { /* If not null string, not [stack] or [vdso] */
>         imagefd = mtcp_sys_open (area.name, O_RDONLY, 0);
>         if (imagefd < 0) imagefd = 0;
>         else area.flags ^= MAP_ANONYMOUS;
>       }
> 
> mtcp_sys_mmap (area.addr, area.size, area.prot | PROT_WRITE,
>                  area.flags, imagefd, area.offset);
> 
> Does anybody know why these files are loaded into physical memory and become
> dirty?
> 
> Thanks,
> Ying

> ------------------------------------------------------------------------------
> Achieve unprecedented app performance and reliability
> What every C/C++ and Fortran developer should know.
> Learn how Intel has extended the reach of its next-generation tools
> to help boost performance applications - inlcuding clusters.
> http://p.sf.net/sfu/intel-dev2devmay

> _______________________________________________
> Dmtcp-forum mailing list
> Dmt...@li...
> https://lists.sourceforge.net/lists/listinfo/dmtcp-forum