Re: [Pas-dev] Apache, mod_perl and Preloading?

[Kyle R . B. <mo...@vo...>]

> True, but there are some details you're forgetting. :) Even get/sets
> arent going to avoid the whole copy on write overhead. At least I dont
> think so.... and I dont have time right now to check.

The only benefit would be that the shared (compiled) code (the compiled
Perl code, the op trees) would be shared.  Per-child data segments (member
variables as you point out) would be subject to copy-on-write and diverge
once any 'set' had been invoked.  That's unavoidable and expected.

> My real question is... are you SURE that this will help? Wont execute()
> be different as far as variables go on a per process basis? There really
> isnt a way to force constants to use one page and variables to use
> another..... Even if the code starts out shared, most page objects will
> wind up being unshared simply by their nature. The lower, core modules
> are more likely to stay shared longer. So preloading and lowering
> maxrequests per child may help if done in tandem.

This should work for the same reason preloading core modules works --
code is not modified so it stays shared (no write, no copy-on-write)
data that never changes (like all those strings that represent the
bulk of the HTML in the psp files) will stay shared as well since
those stirngs are effectivly constant across the lifetime of the 
appilcation.

> Still, I havent spent too much time looking at the guts of apache memory
> management. I've found that simply by following the tuning guide and
> loading the common core modules performance is fine.

I don't think it's 100% Apache we're talking about here, the whole
copy-on-write feature is part of the process model on most unixes
for children that are the result of a fork (unless there is a subsequnt
exec call used to overwrite the process).

> I do concur we would benefit from a build spider to precompile all the
> psp's, but honestly StatINC is more overhead. 

I'm not sure I understandt his - StatINC is more overhead than what?

> Before you kill yourself doing this, check out this link:
> http://perl.apache.org/docs/1.0/guide/performance.html#Know_Your_Operating_System
> 
> I could just plagerize it, but I'd probably just explain it wrong ;)

Quoting that page:

  Do not forget that if you preload most of your code at server 
  startup, the newly forked child gets ready very fast, because 
  it inherits most of the preloaded code and the perl interpreter 
  from the parent process.

And additaionly that page says that even though children will grow
their memory usage as the number of requests they service climbs,
it generaly says that it's still worth preloading as much of the
codebase as possible.

So even though it's not a be-all end-all win, it's still a big win
to preload.

> Granted its not written for pas, but optimizing at the level you're
> talking about (OS tuning) is certainly addressed. 
> 
> Whats the performance issue you're seeing thats prompting you to go down
> this path to begin with? I'm just asking so I can understand where
> you're coming from.

I want the memory usage of our site to be lower.  For our dev envronment
we've got 3 apache instances set up so we can run 3 applications and
keep them seperate from each other.  All those children and all that
memory adds up to swapping (the same box runs the 3 databases as well).
Our production environment will soon be just like this.  Memory reduciton
allows us to do more with less hardware.

> I wish I had more time to help with this, its an interesting problem and
> there's some great tools to gather data...... 


My conclusions from our conversation and the page you referenced are:
  - at a minimum pre-compilation and pre-loading reduces the time 
    it takes a newly forked child to service requests for a given page.
    In the non-pre-load scenario, the child may have to perform 
    PSP compliation, and it may have to perform Perl code compilation.
  - at a minimum pre-loading helps share at least some memory, the
    benefits are maximized if you keep the MaxRequestsPerChild down
    to a reasonable level to flush memory that resulted in copy-on-write
    because of member variables and per-request handling duties.

So, overall, pre-compliation and pre-loading at Apache startup time are
worth the effort of writing a new startup.pl - right?


Kyle

-- 

------------------------------------------------------------------------------
Wisdom and Compassion are inseparable.
        -- Christmas Humphreys
mo...@vo...                            http://www.voicenet.com/~mortis
------------------------------------------------------------------------------