Re: [Firebird-devel] Non-blocking Memory Allocator

[Alex P. <pe...@in...>]

Jim Starkey wrote:

>> What about small blocks - let's fix the fact, that this memory is 
>> never returned to OS before firebird shutdown.
>>
> 
> Alex, where is the gain?  If there isn't a benefit, what's the point?
> 

Sorry - may be I badly expressed what I wanted in English. I wanted to 
say, that we don't return and don't plan to return small memory blocks 
to OS, and this is an official feature of our memory allocator.

> No, everything concerning malloc() is not true of Win32.  As far as I 
> know, the only operating systems that support non-contiguous virtual 
> address space are those designed by D. N. Cutler: VMS, Vax/ELN, and NT.  
> Vax/ELN perform interprocess communication by tweaking the hardware 
> memory map.
> 

 From linux man:
"The munmap system call deletes the mappings for the specified address 
range, and causes further references to addresses within the range to 
generate invalid memory references."

Munmap() is posix compliant system call, supported at least by linux and 
freebsd. Therefore I think you are a bit wrong here - modern *nixes do 
support non-contiguous virtual address space.

> I think you will find that deallocate and reallocating chunks of virtual 
> memory is one of the most expensive form of memory management known to 
> operating systems, require switching to kernel mode, finding suitable 
> pieces of the address space to use, interlocking the appropriate parts 
> of the OS paging tables, etc.  And for what possible gain?  The primary 
> use of Firebird is as a server process that is going to use memory and 
> die.  Playing ping pong with virtual address segments is totally wasted 
> cycles.  If you we primarily concerned about classic, we could have a 
> discussion.  But central server mode?  Lots of pain, no gain.
> 

I know a lot of people, who do not use firebird on a dedicated server (I 
mean production use). I'll estimate that at least half of our clients do 
not have (and for various reasons do not *want* to have) dedicated 
server for firebird.
In this case ability to return at least memory, used by database buffer, 
is very useful.

>> 2. May be it's worth use direct OS allocation for large blocks (like 
>> DB cache)? If we can really return that memory to OS, this makes sense 
>> for me.
> 
> 
> You still haven't established a case for this.
> 

It seems to me I've already established it in my previous letter 
(non-dedicated box with firebird), but you have removed it from your 
letter. As far as I remember, interbase was for a long time positioned 
as scalable solution - from single computer up to a big dedicated 
server. On the low end of that range ability to return memory to OS is 
very useful. I don't suggest to do impossible (or almost impossible 
without bi-i-i-ig overhead) things, like collecting small blocks back to 
big ones, but in case of, for example, DB cache - whats wrong with it?

Alex.