Menu

PyFoam Merge Request #1: Possible overestimation in memory usage for linux (open)

Merging...

Merged

Something went wrong. Please, merge manually

Checking if merge is possible...

Something went wrong. Please, merge manually

Vinicius Pessoa Mapelli wants to merge 1 commit from /u/vmapelli/openfoam-extend/ to develop, 2024-09-27

Hello. I have been using pyFoam for carrying out some parametric studies and I faced some inconsistent results using distinct memory usage metrics. After reading this code and reading some about memory benchmarking, I think the algorithm implemented is possibly overestimating memory usage. From linux kernel documentation (https://docs.kernel.org/filesystems/proc.html):
"
The “proportional set size” (PSS) of a process is the count of pages it has in memory, where each page is divided by the number of processes sharing it. So if a process has 1000 pages all to itself, and 1000 shared with one other process, its PSS will be 1500. “Pss_Dirty” is the portion of PSS which consists of dirty pages. (“Pss_Clean” is not included, but it can be calculated by subtracting “Pss_Dirty” from “Pss”.)
"
In that way, PSS already take into account private memory usage, such that we need to sum only PSS values to get current the memory usage of a parallel run. Psutil library also implies this same behavior to me.

In my tests, I have been reading from /proc/pid/smaps_rollup, which is the same method that psutil library use when running on a kernel with this feature (https://github.com/giampaolo/psutil/blob/master/psutil/_pslinux.py#L1978). So, in my tests, this modification make the results consistent again. Thus, I am creating this merge request for your evaluation.

Thank you for your time and this wonderful library.

Commit Date  
[6ba236] (developtip) by Vinicius P. Mapelli Vinicius P. Mapelli

Sum only proportional set size for multiprocess runs

2024-09-27 00:05:17 Tree

Discussion


Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.