[Openinteract-dev] Preloading of everything

[<an...@io...>]

Hello everyone :) It's been pretty quiet lately, but at least our 
project with OI2 has not stopped. I personally have grown to like OI2 
more and more as I use it and would like to start using it also on some 
smaller projects than Dicole. For smaller projects it is not however 
very easy to justify the amount OI2 requires memory and I would like to 
touch the issue of preloading everything at startup. There are at least 
two problems with preloading everything:

1. At the moment, OI2 is completely unusable as CGI because it takes 
several seconds on a fast machine to initialize Context fully. This is a 
shame since otherwise OI2 works with CGI as well as it does on mod_perl, 
FastCGI and the daemon.

2. Multiple installations of OI2 require that they are running on 
different Apaches, which means completely different perl interpreters, 
which means multiplication of all the code in memory. This could be 
acceptable if the memory footprint was not so huge and huge amounts of 
it were not polluted during the first request.

On single, heavily resourced installations the current approach works - 
and works very nicely. But with smaller resources, limited environment 
or multiple installations this might not always be the best option.

Thus I think that it would be nice to have an option not to fully 
initialize the Context but still maintain it fully functional on 
request. On many occasions trading speed for lower memory usage or 
faster startup time could be welcome.

It is at the moment already possible to easily leave some portions of 
Context uninitialized because this was needed to speed up oi2_manage, 
but at the moment this means also that some of the functionality is lost.

One major time & space hog is preloading of all actions. This is done 
for most actions already when the action table is assigned to CTX since 
we need to gather actions dispatch url data (and thus requiring them 
separately afterwards seems to be useless?). Could we gather this 
dispatch url data (or the whole action table?) separately before Context 
initialization and place it in a "cached" repository? Normal mod_perl 
server start could invoke this generation before startup, but for CGI 
this could be done by hand after each setup/override modifications or 
package installs. Action's init_at_startup() would probably also need to 
be rethought.

Other noteworthy task is preloading of SPOPS objects. In fact it seems 
that at the moment you can already skip InitializeSPOPS and things "just 
work" :) (I haven't tested this yet). This just leaves the gathering of 
all spops config data which could be handled similarily as the action table.

Also one could leave some languages unintialized even though this isn't 
_currently_ a major timesaver. It will be one in the future since I'm 
pretty sure we are going to have way over 10 different localizations 
eventually. This I have not investigated much but could guess that it 
would be possible to actually pregenerate the classes that OI2 now 
generates and evaluates at startup and require them only when needed.

So is there some fundamental problem that I have missed that makes some 
of this impossible? To me it seems that the OI2 infrastructure is so 
well built that this could be done with relatively small changes, but 
since I can't say that I see the real picture perfectly, I would welcome 
some input from Chris - at least in theoretical level ;)

When it comes to managing different types on initializations I think we 
could use the recently added dependency builder. Instead of running all 
setup actions by default, one could specify what setup actions are run 
and then everything that those packages are dependent on would also run. 
One could then craft different metapackages with dependencies for 
different types of startups.

What do you all think about this? Am I the only one who would like a 
"light weight" OI2?

- Antti