openinteract-dev

[Openinteract-dev] OI 2 testing

[Andreas N. <an...@kl...>]

Hey folks,

finally found the time to do some OI 2 / Beta 0 testing.

First impression in short: already works great.

Testenvironment was RedHat Linux 9, which includes a 5.8.0-ithread perl
version. During the installation, I only had the "usual" RH 9 and perl
problems comming from the new defaults for languages et. al ( e.g. UTF-8
charsets ), which lead to problems with quite a few perl moduels.

Apart from this, I only got an error during make test from the SQLite
module, which I chose to ignore.

After this I followed the readme / Manual::Intro and setup a testing
site with lwp. This all worked by the book !

The only spottable "error" I found in the news package: here the display
of dates is not correct in the templates. It shows the class name, so I
assume a deref is missing somewhere.

Switching on debugging als shows, that OI 2 is whining about a few
classes not specifying a "date conversion format". These are classes
News, ErrorObject, Page, User and ObjectAction.

Next I tested with apache 1.3.26 / mod_perl 1.27 by switching the types
in the configs. I was interested, if this would do any harm to the site
or change anything in the layout - no - works w/o a problem.

Now I switched the DBI type from SQLite to Mysql 4.0.13, reinstalled the
SQL etc -> success. 

Now back to LWP -> no problem.

Since this all worked to easy so far ;-), I tried to go for
Apache2/mod_perl2 by registering a new factory type for the
request/response classes and setting up an Apache2::OpenInteract2
handler class. Basically, it turns out to be a rather small task with
the main "problem" being, that the mod_perl2 documentation is not ready
yet. 

BTW: could anyone tell me where $REQUEST_CLASS is set in Request.pm and
where the get_current method in Request/Apache.pm comes from ?

I fell across s.th. different, though: treads and databases !
Since mod_perl2 works with ithreads ( meaning one perl instance in every
thread), which means, a shared nothing aproach is used, I did not see
reason for not trying the treaded worker mpm. 
The problem is, that in perl 5.8.0 you cannot share any object across
threads, so also no db connections. This also means, that you cannot use
any db object from the initial thread in any other thread.
Here I fell with OI2, since the db connections are made quite early,
before the initial thread is cloned to handle the request.

Probably, you would have the same problem with any other threaded
environment, as long as the db connections are setup in advance.

Is there a chance to kick the connections and reestablish them in them
when the request is handled? At that point, you could do caching of db
connections, since you are within a separate thread.


Also I tried to read a bit through the OI2 code and find it a lot more
understandable than the OI1 code. I especially like the use of
Class::Accessor and Class::Factory and the CTX singleton.
Looking at the new News package, I found it not too different from the
OI1 packages, which is great because it means that porting modules
should not hurt too much.
The verbose feedback of oi2_manage is also a good idea.

In short: first impression is really great - well done Chris !

regards,

Andreas

Re: [Openinteract-dev] OI 2 testing

[Chris W. <ch...@cw...>]

Andreas Nolte wrote:
> finally found the time to do some OI 2 / Beta 0 testing.
>
> First impression in short: already works great.

This is extremely valuable. Thanks a ton for taking the time.

> After this I followed the readme / Manual::Intro and setup a testing
> site with lwp. This all worked by the book !

Cool!

> The only spottable "error" I found in the news package: here the display
> of dates is not correct in the templates. It shows the class name, so I
> assume a deref is missing somewhere.

I'll check into this. All date fields in core packages are converted 
automatically to DateTime objects....

> Switching on debugging als shows, that OI 2 is whining about a few
> classes not specifying a "date conversion format". These are classes
> News, ErrorObject, Page, User and ObjectAction.

...which is why this was complaining. I'll probably just put the 
universal 'yyyy-mm-dd hh:mm:ss' in for this: I think any database 
should be able to handle it.

> Next I tested with apache 1.3.26 / mod_perl 1.27 by switching the types
> in the configs. I was interested, if this would do any harm to the site
> or change anything in the layout - no - works w/o a problem.

hooray!

> Now I switched the DBI type from SQLite to Mysql 4.0.13, reinstalled the
> SQL etc -> success. 

Even better -- I'd only tested with PostgreSQL (7.3 something) and 
SQLite.

> Since this all worked to easy so far ;-), I tried to go for
> Apache2/mod_perl2 by registering a new factory type for the
> request/response classes and setting up an Apache2::OpenInteract2
> handler class. Basically, it turns out to be a rather small task with
> the main "problem" being, that the mod_perl2 documentation is not ready
> yet. 

You're very optimistic :-)

> BTW: could anyone tell me where $REQUEST_CLASS is set in Request.pm and
> where the get_current method in Request/Apache.pm comes from ?

During setup OI2::Context->assign_request_type() (and 
assign_response_type() for Response.pm) calls 
OI2::Request->set_implementation_type() with the relevant type name. 
The type name is read from the server.ini file, although you can set 
it afterward too.

> I fell across s.th. different, though: treads and databases !
> Since mod_perl2 works with ithreads ( meaning one perl instance in every
> thread), which means, a shared nothing aproach is used, I did not see
> reason for not trying the treaded worker mpm. 
> The problem is, that in perl 5.8.0 you cannot share any object across
> threads, so also no db connections. This also means, that you cannot use
> any db object from the initial thread in any other thread.
> Here I fell with OI2, since the db connections are made quite early,
> before the initial thread is cloned to handle the request.
 > ...
 > Is there a chance to kick the connections and reestablish them
 > in them when the request is handled? At that point, you could do
 > caching of db connections, since you are within a separate thread.

I had to deal with this in the LWP daemon too. Before we do the fork 
we execute the following:

     # We need to close all database handle created in the
     # initialization process so the child doesn't try to use it.
     OpenInteract2::DatasourceManager->shutdown;

In theory this would work with mp2 as well -- you'd probably execute 
this as the last line in the startup.pl.

After seeing Artur's presentation on threads at YAPC::NA I'm curious 
to see what other changes are necessary. I'm in the process of 
modifying OI2::Context so that certain configuration items (action 
table, SPOPS config info, ...) are accessed by methods alone... This 
will make it easier to follow what's going on and also provide the 
means to use threads::shared to share what little read/write data 
exists in the CTX object.

> Also I tried to read a bit through the OI2 code and find it a lot more
> understandable than the OI1 code. I especially like the use of
> Class::Accessor and Class::Factory and the CTX singleton.

Yeah -- there shouldn't be anymore magical hashref accesses :-)

> Looking at the new News package, I found it not too different from the
> OI1 packages, which is great because it means that porting modules
> should not hurt too much.

It really shouldn't be too bad -- most everything is:

   - changing $R data access/methods to CTX/request/response
   - accessing action state using ->param(...) rather than passing 
around \%p
   - calling 'generate_content()' on the action itself and passing 
only parameter and template identifier info
   - if you're using OI::CommonHandler you'll need to move most/all 
of the declarations into the action configuration, change some 
method names (e.g., 'show' becomes 'display', 'display_form' and 
'display_add'; 'edit' becomes 'update' and 'add', etc.) and modify 
what you inherit from

> The verbose feedback of oi2_manage is also a good idea.

I think I'm going to move most of the functionality from oi2_manage 
into a class so packages can declare tasks at runtime and allow 
addtional parameters to be passed in... we'll see. It's MUCH easier 
now to add tasks, report info, etc.

> In short: first impression is really great - well done Chris !

Thanks! The real test comes when you try to port NATS and other 
modules :-)

Chris

-- 
Chris Winters (ch...@cw...)
Building enterprise-capable snack solutions since 1988.

Re: [Openinteract-dev] OI 2 testing

[Andreas N. <an...@kl...>]

Hi Chris,

the tip with for the thread problem (
OpenInteract2::DatasourceManager->shutdown; )
seams to work ! Now it` s up to porting the Request/Response methods.
I`ll take the approach to use the mp2 api where available and use some
std. module for the rest ( cookies, uploads etc ) and not
libapreq-2-alpha .

regards,

Andreas


On Mon, 2003-06-23 at 08:09, Chris Winters wrote:
> Andreas Nolte wrote:
> > finally found the time to do some OI 2 / Beta 0 testing.
> >
> > First impression in short: already works great.
> 
> This is extremely valuable. Thanks a ton for taking the time.
> 
> > After this I followed the readme / Manual::Intro and setup a testing
> > site with lwp. This all worked by the book !
> 
> Cool!
> 
> > The only spottable "error" I found in the news package: here the display
> > of dates is not correct in the templates. It shows the class name, so I
> > assume a deref is missing somewhere.
> 
> I'll check into this. All date fields in core packages are converted 
> automatically to DateTime objects....
> 
> > Switching on debugging als shows, that OI 2 is whining about a few
> > classes not specifying a "date conversion format". These are classes
> > News, ErrorObject, Page, User and ObjectAction.
> 
> ...which is why this was complaining. I'll probably just put the 
> universal 'yyyy-mm-dd hh:mm:ss' in for this: I think any database 
> should be able to handle it.
> 
> > Next I tested with apache 1.3.26 / mod_perl 1.27 by switching the types
> > in the configs. I was interested, if this would do any harm to the site
> > or change anything in the layout - no - works w/o a problem.
> 
> hooray!
> 
> > Now I switched the DBI type from SQLite to Mysql 4.0.13, reinstalled the
> > SQL etc -> success. 
> 
> Even better -- I'd only tested with PostgreSQL (7.3 something) and 
> SQLite.
> 
> > Since this all worked to easy so far ;-), I tried to go for
> > Apache2/mod_perl2 by registering a new factory type for the
> > request/response classes and setting up an Apache2::OpenInteract2
> > handler class. Basically, it turns out to be a rather small task with
> > the main "problem" being, that the mod_perl2 documentation is not ready
> > yet. 
> 
> You're very optimistic :-)
> 
> > BTW: could anyone tell me where $REQUEST_CLASS is set in Request.pm and
> > where the get_current method in Request/Apache.pm comes from ?
> 
> During setup OI2::Context->assign_request_type() (and 
> assign_response_type() for Response.pm) calls 
> OI2::Request->set_implementation_type() with the relevant type name. 
> The type name is read from the server.ini file, although you can set 
> it afterward too.
> 
> > I fell across s.th. different, though: treads and databases !
> > Since mod_perl2 works with ithreads ( meaning one perl instance in every
> > thread), which means, a shared nothing aproach is used, I did not see
> > reason for not trying the treaded worker mpm. 
> > The problem is, that in perl 5.8.0 you cannot share any object across
> > threads, so also no db connections. This also means, that you cannot use
> > any db object from the initial thread in any other thread.
> > Here I fell with OI2, since the db connections are made quite early,
> > before the initial thread is cloned to handle the request.
>  > ...
>  > Is there a chance to kick the connections and reestablish them
>  > in them when the request is handled? At that point, you could do
>  > caching of db connections, since you are within a separate thread.
> 
> I had to deal with this in the LWP daemon too. Before we do the fork 
> we execute the following:
> 
>      # We need to close all database handle created in the
>      # initialization process so the child doesn't try to use it.
>      OpenInteract2::DatasourceManager->shutdown;
> 
> In theory this would work with mp2 as well -- you'd probably execute 
> this as the last line in the startup.pl.
> 
> After seeing Artur's presentation on threads at YAPC::NA I'm curious 
> to see what other changes are necessary. I'm in the process of 
> modifying OI2::Context so that certain configuration items (action 
> table, SPOPS config info, ...) are accessed by methods alone... This 
> will make it easier to follow what's going on and also provide the 
> means to use threads::shared to share what little read/write data 
> exists in the CTX object.
> 
> > Also I tried to read a bit through the OI2 code and find it a lot more
> > understandable than the OI1 code. I especially like the use of
> > Class::Accessor and Class::Factory and the CTX singleton.
> 
> Yeah -- there shouldn't be anymore magical hashref accesses :-)
> 
> > Looking at the new News package, I found it not too different from the
> > OI1 packages, which is great because it means that porting modules
> > should not hurt too much.
> 
> It really shouldn't be too bad -- most everything is:
> 
>    - changing $R data access/methods to CTX/request/response
>    - accessing action state using ->param(...) rather than passing 
> around \%p
>    - calling 'generate_content()' on the action itself and passing 
> only parameter and template identifier info
>    - if you're using OI::CommonHandler you'll need to move most/all 
> of the declarations into the action configuration, change some 
> method names (e.g., 'show' becomes 'display', 'display_form' and 
> 'display_add'; 'edit' becomes 'update' and 'add', etc.) and modify 
> what you inherit from
> 
> > The verbose feedback of oi2_manage is also a good idea.
> 
> I think I'm going to move most of the functionality from oi2_manage 
> into a class so packages can declare tasks at runtime and allow 
> addtional parameters to be passed in... we'll see. It's MUCH easier 
> now to add tasks, report info, etc.
> 
> > In short: first impression is really great - well done Chris !
> 
> Thanks! The real test comes when you try to port NATS and other 
> modules :-)
> 
> Chris

Re: [Openinteract-dev] OI 2 testing

[Chris W. <ch...@cw...>]

Andreas Nolte wrote:
> the tip with for the thread problem (
> OpenInteract2::DatasourceManager->shutdown; )
> seams to work ! Now it` s up to porting the Request/Response methods.
> I`ll take the approach to use the mp2 api where available and use some
> std. module for the rest ( cookies, uploads etc ) and not
> libapreq-2-alpha .

Great! One source of info might be the OpenFrame project. I swiped 
some of the other Request implementations from them (with a nod of 
the head, of course) and IIRC Leon create at least a prelim 
implementation for Apache2.

Chris

-- 
Chris Winters (ch...@cw...)
Building enterprise-capable snack solutions since 1988.