Thread: [Sqlrelay-discussion] Memory consumtion under SunOS (detach or nodetach)

Brought to you by: mused

sqlrelay-discussion

[Sqlrelay-discussion] Memory consumtion under SunOS (detach or nodetach)

From: Renat S. <sr...@st...> - 2010-12-13 13:33:57

Hello!

I found a strange effects while running sqlrelay under SunOS 5.10 
(Solaris). It seams that "detach" option has some influence on memory 
management.

case 1: -detach was applied
---- top output ----
  PID USERNAME LWP PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
3630 onyma    1   59  0     530M   33M sleep    0:00    0.00% 
./sqlr-connection-oracle8 -silent -detach
---- end of top output ----

case 2: -detach wasn't applied
---- top output ----
  PID USERNAME LWP PRI NICE  SIZE   RES STATE    TIME    CPU COMMAND
3647 onyma    1   42  0     532M   468M sleep   0:00    0.00% 
./sqlr-connection-oracle8 -silent -ttl 60
---- end of top output ----

Do you see the difference? RES in the first case is 33M but 468M in the 
second. Do you have any ideas why it is so?

As a result of this I have out of memory problem on our production 
server under SunOS 5.10 (Solaris).

-- 
Renat Sabitov   e-mail: sr...@st...
Stack Soft         jid: sr...@ja...

Re: [Sqlrelay-discussion] Memory consumtion under SunOS (detach or nodetach)

From: Renat S. <sr...@st...> - 2010-12-13 13:34:11

> case 2: -detach wasn't applied

this should be read as

case 2: -nodetach was applied



-- 
Renat Sabitov   e-mail: sr...@st...
Stack Soft         jid: sr...@ja...

Re: [Sqlrelay-discussion] Memory consumtion under SunOS (detach or nodetach)

From: Renat S. <sr...@st...> - 2010-12-13 15:57:35

Further investigations shows, that there are 2 heap segments for 
"nodetach" case, and the second is exactly the size of memory, allocated 
for oracle8 cursors buffers (def_buf). It is 5 (cursors) * 10 
(FETCH_AT_ONCE) * 256 (MAX_SELECT_LIST_SIZE) * 32768 
(MAX_ITEM_BUFFER_SIZE) = 409600K. Ok, but why it goes in whole piece to 
"Anon" memory, which is "the actual physical memory that is allocated 
uniquely to my process and not to any other processes" (according to 
http://blogs.sun.com/nickstephen/entry/solaris_measuring_the_memory_overhead)? 


pmem output:

case 1 (nodetach):

  Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
00010000      56      56       -       - r-x--  sqlr-connection-oracle8
0002C000      16      16       8       - rwx--  sqlr-connection-oracle8
00030000    3904    1248    1248       - rwx--    [ heap ]
00400000  409600  409600  409600       - rwx--    [ heap ]
F7000000   85128   51400       -       - r-x--  libociei.so
FC330000       8       8       8       - rwx--  libociei.so
FC400000     888     888       -       - r-x--  libc.so.1
...

case 2 (detach):

  Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
00010000      56      56       -       - r-x--  sqlr-connection-oracle8
0002C000      16      16       8       - rwx--  sqlr-connection-oracle8
00030000  411488   15296   14288       - rwx--    [ heap ]
F7000000   85128   51400       -       - r-x--  libociei.so
FC330000       8       8       -       - rwx--  libociei.so
FC400000     888     888       -       - r-x--  libc.so.1
...


-- 
Renat Sabitov   e-mail: sr...@st...
Stack Soft         jid: sr...@ja...

Re: [Sqlrelay-discussion] Memory consumtion under SunOS (detach or nodetach) - SOLUTION

From: Renat S. <sr...@st...> - 2010-12-14 15:29:46

Having made all sorts of tests, I found that Solaris dynamically chooses 
pagesize for heap. In addition to this, malloc on solaris writes some 
data in memory (I assume it creates a pool and stores some information 
of allocated size in it), so all "dirty" chunks became an "Anon" memory. 
When it decides to use 4M pagesize, writing only few bytes makes 4M 
chunk to be stored in "real" memory of process.

I found some information about controlling pagesize for process.

http://www.solarisinternals.com/wiki/index.php/Multiple_Page_Size_Support#Support_for_Large_Pages_within_the_Kernel.27s_Address_Space

You can set preferred pagesize by 3 ways:
   1. A wrapper program, ppgsz(1)
   2. A preload library, libmpss.so.1
   3. A programmatic interface, memcntl(2)

We are particularly interested in the second way, because of it's 
simplicity. First way won't work, since "exec" function returns pagesize 
choosing algorithm to default. So having this commands in the start 
script will keep pagesize to 8K

export LD_PRELOAD=$LD_PRELOAD:mpss.so.1
export MPSSHEAP=8K

13.12.2010 18:57, Renat Sabitov пишет:
> Further investigations shows, that there are 2 heap segments for
> "nodetach" case, and the second is exactly the size of memory, allocated
> for oracle8 cursors buffers (def_buf). It is 5 (cursors) * 10
> (FETCH_AT_ONCE) * 256 (MAX_SELECT_LIST_SIZE) * 32768
> (MAX_ITEM_BUFFER_SIZE) = 409600K. Ok, but why it goes in whole piece to
> "Anon" memory, which is "the actual physical memory that is allocated
> uniquely to my process and not to any other processes" (according to
> http://blogs.sun.com/nickstephen/entry/solaris_measuring_the_memory_overhead)?
>
>
> pmem output:
>
> case 1 (nodetach):
>
>    Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
> 00010000      56      56       -       - r-x--  sqlr-connection-oracle8
> 0002C000      16      16       8       - rwx--  sqlr-connection-oracle8
> 00030000    3904    1248    1248       - rwx--    [ heap ]
> 00400000  409600  409600  409600       - rwx--    [ heap ]
> F7000000   85128   51400       -       - r-x--  libociei.so
> FC330000       8       8       8       - rwx--  libociei.so
> FC400000     888     888       -       - r-x--  libc.so.1
> ...
>
> case 2 (detach):
>
>    Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
> 00010000      56      56       -       - r-x--  sqlr-connection-oracle8
> 0002C000      16      16       8       - rwx--  sqlr-connection-oracle8
> 00030000  411488   15296   14288       - rwx--    [ heap ]
> F7000000   85128   51400       -       - r-x--  libociei.so
> FC330000       8       8       -       - rwx--  libociei.so
> FC400000     888     888       -       - r-x--  libc.so.1
> ...
>
>


-- 
Renat Sabitov   e-mail: sr...@st...
Stack Soft         jid: sr...@ja...

Re: [Sqlrelay-discussion] [SPAM] - Re: Memory consumtion under SunOS (detach or nodetach) - SOLUTION - Email found in subject

From: Carlos V. <cve...@em...> - 2010-12-14 15:50:24

Nice work Renat!

-----Original Message-----
From: Renat Sabitov [mailto:sr...@st...] 
Sent: Tuesday, December 14, 2010 9:30 AM
To: Discussion of topics related to SQL Relay
Subject: [SPAM] - Re: [Sqlrelay-discussion] Memory consumtion under SunOS (detach or nodetach) - SOLUTION - Email found in subject

Having made all sorts of tests, I found that Solaris dynamically chooses 
pagesize for heap. In addition to this, malloc on solaris writes some 
data in memory (I assume it creates a pool and stores some information 
of allocated size in it), so all "dirty" chunks became an "Anon" memory. 
When it decides to use 4M pagesize, writing only few bytes makes 4M 
chunk to be stored in "real" memory of process.

I found some information about controlling pagesize for process.

http://www.solarisinternals.com/wiki/index.php/Multiple_Page_Size_Support#Support_for_Large_Pages_within_the_Kernel.27s_Address_Space

You can set preferred pagesize by 3 ways:
   1. A wrapper program, ppgsz(1)
   2. A preload library, libmpss.so.1
   3. A programmatic interface, memcntl(2)

We are particularly interested in the second way, because of it's 
simplicity. First way won't work, since "exec" function returns pagesize 
choosing algorithm to default. So having this commands in the start 
script will keep pagesize to 8K

export LD_PRELOAD=$LD_PRELOAD:mpss.so.1
export MPSSHEAP=8K

13.12.2010 18:57, Renat Sabitov пишет:
> Further investigations shows, that there are 2 heap segments for
> "nodetach" case, and the second is exactly the size of memory, allocated
> for oracle8 cursors buffers (def_buf). It is 5 (cursors) * 10
> (FETCH_AT_ONCE) * 256 (MAX_SELECT_LIST_SIZE) * 32768
> (MAX_ITEM_BUFFER_SIZE) = 409600K. Ok, but why it goes in whole piece to
> "Anon" memory, which is "the actual physical memory that is allocated
> uniquely to my process and not to any other processes" (according to
> http://blogs.sun.com/nickstephen/entry/solaris_measuring_the_memory_overhead)?
>
>
> pmem output:
>
> case 1 (nodetach):
>
>    Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
> 00010000      56      56       -       - r-x--  sqlr-connection-oracle8
> 0002C000      16      16       8       - rwx--  sqlr-connection-oracle8
> 00030000    3904    1248    1248       - rwx--    [ heap ]
> 00400000  409600  409600  409600       - rwx--    [ heap ]
> F7000000   85128   51400       -       - r-x--  libociei.so
> FC330000       8       8       8       - rwx--  libociei.so
> FC400000     888     888       -       - r-x--  libc.so.1
> ...
>
> case 2 (detach):
>
>    Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
> 00010000      56      56       -       - r-x--  sqlr-connection-oracle8
> 0002C000      16      16       8       - rwx--  sqlr-connection-oracle8
> 00030000  411488   15296   14288       - rwx--    [ heap ]
> F7000000   85128   51400       -       - r-x--  libociei.so
> FC330000       8       8       -       - rwx--  libociei.so
> FC400000     888     888       -       - r-x--  libc.so.1
> ...
>
>


-- 
Renat Sabitov   e-mail: sr...@st...
Stack Soft         jid: sr...@ja...

------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
Sqlrelay-discussion mailing list
Sql...@li...
https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion
�[�����[�[^KY\��\��[

Re: [Sqlrelay-discussion] Memory consumtion under SunOS (detach or nodetach) - SOLUTION

From: David M. <dav...@fi...> - 2010-12-16 06:27:01

Ok, let me see if I understand this correctly...

Detach instance:  1 heap containing both the result set buffer and 
everything else, with a small page size (4k, 8k, or something)
Nodetach instance: 2 heaps, one containing the result set buffer, with a 
4M page size and another containing everything else with a small page size.

Does that sound right?

Not detaching must trigger some subtle behavior in the solaris memory 
allocator, resulting in the two heaps.  I'm not sure why you wouldn't 
get that in both cases, but I'd think one would have to really 
understand the allocator inside and out to be able to say why.

It sounds like the approach you're using should yield predictable 
results in both cases.  I'd think a second heap with a 4M page size for 
the result set buffer would yield higher performance, but of course, at 
the cost of using additional memory.

This is very interesting, I had no idea solaris did this, or that you 
could override the memory allocator like that.  Very cool.

Dave
dav...@fi...

On 12/14/2010 10:29 AM, Renat Sabitov wrote:
> Having made all sorts of tests, I found that Solaris dynamically chooses
> pagesize for heap. In addition to this, malloc on solaris writes some
> data in memory (I assume it creates a pool and stores some information
> of allocated size in it), so all "dirty" chunks became an "Anon" memory.
> When it decides to use 4M pagesize, writing only few bytes makes 4M
> chunk to be stored in "real" memory of process.
>
> I found some information about controlling pagesize for process.
>
> http://www.solarisinternals.com/wiki/index.php/Multiple_Page_Size_Support#Support_for_Large_Pages_within_the_Kernel.27s_Address_Space
>
> You can set preferred pagesize by 3 ways:
>     1. A wrapper program, ppgsz(1)
>     2. A preload library, libmpss.so.1
>     3. A programmatic interface, memcntl(2)
>
> We are particularly interested in the second way, because of it's
> simplicity. First way won't work, since "exec" function returns pagesize
> choosing algorithm to default. So having this commands in the start
> script will keep pagesize to 8K
>
> export LD_PRELOAD=$LD_PRELOAD:mpss.so.1
> export MPSSHEAP=8K
>
> 13.12.2010 18:57, Renat Sabitov пишет:
>    
>> Further investigations shows, that there are 2 heap segments for
>> "nodetach" case, and the second is exactly the size of memory, allocated
>> for oracle8 cursors buffers (def_buf). It is 5 (cursors) * 10
>> (FETCH_AT_ONCE) * 256 (MAX_SELECT_LIST_SIZE) * 32768
>> (MAX_ITEM_BUFFER_SIZE) = 409600K. Ok, but why it goes in whole piece to
>> "Anon" memory, which is "the actual physical memory that is allocated
>> uniquely to my process and not to any other processes" (according to
>> http://blogs.sun.com/nickstephen/entry/solaris_measuring_the_memory_overhead)?
>>
>>
>> pmem output:
>>
>> case 1 (nodetach):
>>
>>     Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
>> 00010000      56      56       -       - r-x--  sqlr-connection-oracle8
>> 0002C000      16      16       8       - rwx--  sqlr-connection-oracle8
>> 00030000    3904    1248    1248       - rwx--    [ heap ]
>> 00400000  409600  409600  409600       - rwx--    [ heap ]
>> F7000000   85128   51400       -       - r-x--  libociei.so
>> FC330000       8       8       8       - rwx--  libociei.so
>> FC400000     888     888       -       - r-x--  libc.so.1
>> ...
>>
>> case 2 (detach):
>>
>>     Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
>> 00010000      56      56       -       - r-x--  sqlr-connection-oracle8
>> 0002C000      16      16       8       - rwx--  sqlr-connection-oracle8
>> 00030000  411488   15296   14288       - rwx--    [ heap ]
>> F7000000   85128   51400       -       - r-x--  libociei.so
>> FC330000       8       8       -       - rwx--  libociei.so
>> FC400000     888     888       -       - r-x--  libc.so.1
>> ...
>>
>>
>>      
>
>    



_______________________________________________________
Unlimited Disk, Data Transfer, PHP/MySQL Domain Hosting
              http://www.doteasy.com

Re: [Sqlrelay-discussion] Memory consumption under SunOS (detach or nodetach) - SOLUTION

From: Renat S. <sr...@st...> - 2010-12-16 13:06:24

16.12.2010 09:26, David Muse пишет:
> Ok, let me see if I understand this correctly...
>
> Detach instance:  1 heap containing both the result set buffer and
> everything else, with a small page size (4k, 8k, or something)
> Nodetach instance: 2 heaps, one containing the result set buffer, with a
> 4M page size and another containing everything else with a small page size.
>
> Does that sound right?
Yes, exactly like you wrote.

> Not detaching must trigger some subtle behavior in the solaris memory
> allocator, resulting in the two heaps.  I'm not sure why you wouldn't
> get that in both cases, but I'd think one would have to really
> understand the allocator inside and out to be able to say why.

I really don't know what happens inside solaris allocator, but I tried a 
simple program like this and commenting out fork() call was changing 
behaviour.

-----
#include <unistd.h>
#include <stdlib.h>

int main (int argc, char **argv)
{
    if (fork()) exit(0);

    char * a[10];
    char * b = (char *)malloc(1000);
    int i;

    for (i=0; i<50; i++) {
       a[i] = (char *)malloc(256 * 32768);
    }
    sleep ( 30);
    for (i=0; i<50; i++) {
       free (a[i]);
    }
    free (b);
    return 0;
}
-----

> It sounds like the approach you're using should yield predictable
> results in both cases.  I'd think a second heap with a 4M page size for
> the result set buffer would yield higher performance, but of course, at
> the cost of using additional memory.

I'm not sure about higher performance, but cost is a lot of additional 
memory, which can cause swapping and huge performance degrade. :)

-- 
Renat Sabitov   e-mail: sr...@st...
Stack Soft         jid: sr...@ja...