[Poshmodule-users] RE: Starting a list at posh.sf.net

[<ste...@fa...>]

> I have limited time as well. However, I have identified POSH as one of
those
> things that can help me write better applications on mod_python, and close
> the gaps between mod_perl and mod_python. I am going to devote some
serious
> time to it.

Great!

> My basic idea is to modify POSH so that two *independent*
> processes/interpreters can access the same spot of shared memory.
>
> Right now, mod_python starts interpreters after Apache has already forked
or
> threaded. Therefore, if POSH is to be used in mod_python, we'll need a way
of
> sharing the objects across two seperate interpreters or processes.
>
> The only other option is to modify mod_python so that it is started
*before*
> apache forks and threads. I'm not too familiar with how exactly mod_perl
> handles this, but I do know that mod_perl uses shared memory to share
modules
> among the children.
>
> Perhaps by providing a way to identify shared memory spots and to access
them
> from independent processes we can bypass the fork dependence by not
relying
> on them having the same address space.

Some thoughts:

The problem of referring to shared memory locations unambiguously from
separate processes is adressed by the "memory handles" implemented by the
SharedMemHandle struct.  POSH uses one shared memory region to maintain the
'global state', including the IDs for all the other shared memory regions,
but this should probably be easy to work around.  I believe the main problem
we're facing is how to refer to locations in non-shared memory from separate
processes.  That is obviously a problem if the processes don't have similar
address spaces, in which case pointers are inappropriate.  To be able to
"reuse" the code in the interpreter, though, we must have a valid type
pointer in every Python object, including those that are allocated in shared
memory.  However, the type objects are normally allocated in the private
address space of the process.  We might have to allocate the type objects in
shared memory, too, to make this work.  Then there's the problem of all the
functions, docstrings, and other type objects referred to by means of plain
pointers in the type objects.  It's kind of a recursive problem...

I think we can assume that the interpreter's data structures and code are
laid out identically in memory for all processes.  When it comes to
dynamically linked modules, though, I'm unsure of what assumptions we can
make.  The nice thing about POSH so far is that it doesn't require a special
version of the interpreter - it would be nice if we could keep it that way.
One idea that might work is to try to emulate the behaviour of any shared
type by means of one special 'proxy' type, much like the ClassType type
emulates any user-defined (old-style) class.  The motivation for that would
be that all shared objects would then have the same type pointer.  The same
problem of how to ensure the same memory location in all processes for that
type object still remains, though.  The easiest way would obviously be to
declare it statically in the interpreter, but then we need to modify that...

Although modifying mod_python so that it ends up farther up in the process
hierachy might be an easier solution for that case, if we can crack this
problem and support a spawn() model of process management, it would be a
great improvement, since it paves the road for a Win32 version, too.

Steffen