Re: [Pas-dev] Apache, mod_perl and Preloading?

[Mental P. <me...@ne...>]

On Fri, 2003-05-02 at 11:03, Kyle R . Burton wrote:
> > True, but there are some details you're forgetting. :) Even get/sets
> > arent going to avoid the whole copy on write overhead. At least I dont
> > think so.... and I dont have time right now to check.
>=20
> The only benefit would be that the shared (compiled) code (the compiled
> Perl code, the op trees) would be shared.  Per-child data segments (membe=
r
> variables as you point out) would be subject to copy-on-write and diverge
> once any 'set' had been invoked.  That's unavoidable and expected.
>=20

Exactly.... so if you have a ton of content.. this is a win.

This is also why Embperl didnt cache content. It only caches the code so
that lots of html doesnt mean lots of memory. Serving static content of
the disk is still fast.....=20

> > My real question is... are you SURE that this will help? Wont execute()
> > be different as far as variables go on a per process basis? There reall=
y
> > isnt a way to force constants to use one page and variables to use
> > another..... Even if the code starts out shared, most page objects will
> > wind up being unshared simply by their nature. The lower, core modules
> > are more likely to stay shared longer. So preloading and lowering
> > maxrequests per child may help if done in tandem.
>=20
> This should work for the same reason preloading core modules works --
> code is not modified so it stays shared (no write, no copy-on-write)
> data that never changes (like all those strings that represent the
> bulk of the HTML in the psp files) will stay shared as well since
> those stirngs are effectivly constant across the lifetime of the=20
> appilcation.

> > Still, I havent spent too much time looking at the guts of apache memor=
y
> > management. I've found that simply by following the tuning guide and
> > loading the common core modules performance is fine.
>=20
> I don't think it's 100% Apache we're talking about here, the whole
> copy-on-write feature is part of the process model on most unixes
> for children that are the result of a fork (unless there is a subsequnt
> exec call used to overwrite the process).
>=20
> > I do concur we would benefit from a build spider to precompile all the
> > psp's, but honestly StatINC is more overhead.=20
>=20
> I'm not sure I understandt his - StatINC is more overhead than what?
>=20

Sorry. Thats what happens when you word salad between other tings....
what I meant was that turning off StatINC fairly significant perfomance
boost in and of itself.=20


> > Before you kill yourself doing this, check out this link:
> > http://perl.apache.org/docs/1.0/guide/performance.html#Know_Your_Operat=
ing_System
> >=20
> > I could just plagerize it, but I'd probably just explain it wrong ;)
>=20
> Quoting that page:
>=20
>   Do not forget that if you preload most of your code at server=20
>   startup, the newly forked child gets ready very fast, because=20
>   it inherits most of the preloaded code and the perl interpreter=20
>   from the parent process.
>=20
> And additaionly that page says that even though children will grow
> their memory usage as the number of requests they service climbs,
> it generaly says that it's still worth preloading as much of the
> codebase as possible.
>=20
> So even though it's not a be-all end-all win, it's still a big win
> to preload.
>=20
> > Granted its not written for pas, but optimizing at the level you're
> > talking about (OS tuning) is certainly addressed.=20
> >=20
> > Whats the performance issue you're seeing thats prompting you to go dow=
n
> > this path to begin with? I'm just asking so I can understand where
> > you're coming from.
>=20
> I want the memory usage of our site to be lower.  For our dev envronment
> we've got 3 apache instances set up so we can run 3 applications and
> keep them seperate from each other.  All those children and all that
> memory adds up to swapping (the same box runs the 3 databases as well).
> Our production environment will soon be just like this.  Memory reduciton
> allows us to do more with less hardware.
>=20

So are you tuning for speed or memory size? Would it be unreasonable to
just cut down max requests per child? Keep httpd's from caching a ton of
content in the first place? There's also Apache::SizeLimit. You can
control how much shared/unshared memory each httpd can have. THat'll
help ya too.


> > I wish I had more time to help with this, its an interesting problem an=
d
> > there's some great tools to gather data......=20
>=20
>=20
> My conclusions from our conversation and the page you referenced are:
>   - at a minimum pre-compilation and pre-loading reduces the time=20
>     it takes a newly forked child to service requests for a given page.
>     In the non-pre-load scenario, the child may have to perform=20
>     PSP compliation, and it may have to perform Perl code compilation.
>   - at a minimum pre-loading helps share at least some memory, the
>     benefits are maximized if you keep the MaxRequestsPerChild down
>     to a reasonable level to flush memory that resulted in copy-on-write
>     because of member variables and per-request handling duties.
>=20
> So, overall, pre-compliation and pre-loading at Apache startup time are
> worth the effort of writing a new startup.pl - right?
>=20

Yup. I'd say load the most common pages, set a semi-restrictive
Apache::SizeLimit ( think I forgot to mention it before, but
Apache::SizeLimit is nice) in your startup.pl, and lower max requests.
Hopefully that'll get you out of swap in the short term. In the long
term it might be worht it to figure out a hybrid psp/html page. Where
large static content gets served off the disk but the dynamic bits are
in a module.....=20



--=20
Mental (Me...@Ne...)

"Shouting at people who, for one reason or another, cannot hear you=20
is mentally-ill behavior -- or evidence of idiots in command."

--George Smith, a virus researcher and columnist for SecurityFocus.=20

CARPE NOCTEM, QUAM MINIMUM CREDULA POSTERO.

GPG public key: http://www.neverlight.com/pas/Mental.asc