#49 Memory leak when compiling routines

[Alan ONeill]

I recently downloaded the VA's VISTA package, which
contains over 17,000 MUMPS routines. I decided to
compile them with a bash script as follows:

for file in *.m; do mumps $file; done

I was the only one logged onto the system and I started
no other processes while the loop was running. While
the loop ran, I also had the system monitor running,
and I was keeping an eye on the memory and swap space
in use. As each routine compiled, the memory in use
increased. When the loop was complete and I exited the
shell (konsole), the memory was not freed up.

I am running Red Hat Linux 9.0 with kernel version
2.4.20-20.9
I am using GT.M V4.4-003 Linux x86

I also downloaded the source code, and I'm willing to
take a look, but it would be very helpful if I had a
starting point. I did find two C routines named
m_zcompile.c and op_zcompile.c in the sr_port
directory, but I don't know if either one of these or
something all together different is where I should
start my search.

Is this problem being researched? If not and if I may
help, with which routine should I begin my investigation?

If anyone needs more info, just let me know.

[Steven Estes]

Logged In: YES
user_id=97877

Alan,

What are you looking at when you say memory was not
released? What statistic shows memory in use? In use where?
I do not know how familiar you are with Linux so please
forgive if I am stating something you already know. Your bash
shell executed a separate mumps process for each compile.
After each compile, the mumps process was destroyed so it
COULD NOT have leaked even if it wanted to. The process
was gone and any storage it allocated was gone with it. The
only leak it could have potentially left behind would have
been shared memory and none of that is used in a compile.

If you are looking at memory usage in a system monitor and
seeing the free list list shrink, you probably see the cache
pages going up and the buffer pages going up. This is normal.
As you read in files (and write out new object files), Linux
keeps these files in storage in case you need them again.
Yes, it uses up memory but it is not a memory leak and the
storage will be reclaimed if it is needed. It is just an efficient
way Linux uses the storage available to it. Use the linux "free"
command to see how storage is allocated. Likely most of your
pages show up in the "cached" column. The "man free"
command will tell you about the output.

Hope this helps.

Steve

[Alan ONeill]

Logged In: YES
user_id=898340

Hi Steve,

Thanks so much for that great explanation. What I was
looking at was the GUI system monitor, which is not showing
me information about cached pages or buffers. After I read
your response, I tried this:

$ free
$ for file in *.m; do mumps $file; done
$ free

I saw the following for the two free commands:

#1:

total used free shared
buffers cached
Mem: 1014752 105332 909420 0
9396 49108
-/+ buffers/cache: 46828 967924
Swap: 2040244 0 2040244

#2:

total used free shared
buffers cached
Mem: 1014752 589792 424960 0
21784 416780
-/+ buffers/cache: 151228 863524
Swap: 2040244 0 2040244

Indeed the numbers under buffers and cached increased
significantly.

I do appreciate your fast and helpful feedback!

Alan