Webware for Python

With talk of a mini-Apache, patches to Application to get it to serve
HTTP, and all that, I thought I'd try a strategy I brought up earlier,
but which no one seemed very excited about: an HTTP-serving adapter. 
I'd been working with BaseHTTPServer already (included with Python), and
it was surprisingly easy.  1 hour later:

  http://www.colorstudy.net/software/webware/HTTPAdapter.py

With a little testing I see no reason this couldn't be included with
Webware, providing the mini-HTTP-server that is sometimes brought up. 
Maybe (maybe) not as elegant as having Application serve HTTP directly,
but it's simple, short (160 lines), and guaranteed not to have any
adverse effect on anything else in Webware, since it touches nothing
else.

One definite bug is reading POST requests with no Content-Length header
-- when I do self.rfile.read(), instead of
self.rfile.read(contentLength), it hangs.  Probably a subtlety of socket
programming -- no Content-Length header might simply be considered
invalid input.

While it runs in a different process from the AppServer (as an Adapter),
for ease of running/installation it would be entirely possible to fork
off that process when the AppServer is started, giving the illusion of a
single HTTP-serving, Python-Application-Serving program.

  Ian

Geoff wrote:

>I would propose using "hasattr" to check for the existence of awake,
>respond, and sleep, and removing the assert statement.  The RuntimeError
>that results could have a much better error message than the assert would
>give us, anyhow.

I think this is a tidier solution that also removes the need for yet
another configuration setting.

Anyway, code should be written to run correctly without assert statements
given that they are removed on an optimized compilation.

...Edmund.

On Wednesday 10 April 2002 01:53 pm, Geoffrey Talvola wrote:
> I would propose using "hasattr" to check for the existence of awake,
> respond, and sleep, and removing the assert statement.  The
> RuntimeError that results could have a much better error message than
> the assert would give us, anyhow.
>
> But I'm curious on Chuck's take on removing the assert statement.  I
> somehow have a feeling that he has an opinion on this, maybe because
> it was discussed before?
>
> - Geoff

Yes, this was discussed and decided before. Here are the points:

- By requiring subclasses of Servlet, we can put methods in Servlet 
that other parts of WebKit (like the AppServer and Application) can 
make use of.

Otherwise, if we decide to add some method like "serverSidePath" in a 
version 0.4 then we basically break every WebKit application that 
wasn't subclassing Servlet.

We would either get breakage every time we added something (when there 
is often a reasonable default implementation), or we could have to make 
these methods into global functions (that would presumably check 
hasattr() first to see if the servlet wanted to "override them").

There is a "myth" among the Python community that requiring inheritance 
is _always_ bad. It is bad if you only rely on one or two methods and 
don't plan on beefing up the de facto base class. It is also bad with 
builtin types in Py <= 2.2 because they couldn't be subclassed.

- There are probably more required methods than you might think. I 
doubt that it's only awake, respond and sleep.

- Since you are already using WebKit, inheriting from Servlet (or a 
subclass is extremely easy):

from WebKit.HTTPServlet import HTTPServlet
class Foo(HTTPServlet):
	pass

- Python provides multiple inheritance so you're not locked into just 
inheriting from WebKit.

- Invoking hasattr() multiple times will be more expensive than one 
isinstance().

- Checking isinstance() guards against the "duplicate module" bug we 
experienced earlier.



It sounds like the assert should be changed to an "if: raise" to be 
more clear and avoid being "optimized out".

-Chuck


>
> > -----Original Message-----
> > From: iron@... [mailto:iron@...]
> > Sent: Monday, April 08, 2002 3:31 PM
> > To: webware-discuss@...
> > Subject: Re: [Webware-discuss] Allowing delegation in servlets
> >
> > On Mon, Apr 08, 2002 at 02:43:37PM -0400, Geoffrey Talvola wrote:
> > > > My first attempt failed because line 183 of ServletFactory is:
> > > >
> > > >     assert issubclass(theClass, Servlet)
> > > >
> > > > Anyway, I'd like to argue for the removal of the assertion in
> > > > ServletFactory, since it is denying use of a very powerful,
> > > > and arguably
> > > > better technique than inheritance. In any case, assertions
> > > > can be optimized
> > > > away in .pyo files, so that the presence of the assertion
> > > > statement is of
> > > > questionable value...
> > >
> > > I think the assertion is there to help out the Webware user
> >
> > who forgets to
> >
> > > have their servlet inherit from Servlet.  I assume that
> >
> > mistake happens
> >
> > > relatively frequently.  I know I've done it once or twice.
> >
> > Without the
> >
> > > assertion (which pinpoints the problem pretty clearly),
> >
> > presumably you would
> >
> > > wind up with a traceback somewhere else in the guts of
> >
> > Appliction.py that
> >
> > > would be less obvious to track down, especially for a casual
> > > user.
> > >
> > > Now you've identified a legitimate use for a servlet not
> >
> > derived from
> >
> > > Servlet.  So you want to remove the assertion, but that
> >
> > would hurt casual
> >
> > > Webware users hwo might forget the base class.  So maybe
> >
> > there should be an
> >
> > > option "AllowServletsNotDerivedFromServlet" that defaults
> >
> > to 0, but you can
> >
> > > just flip the switch and it skips the assertion?  I would
> >
> > imagine this to be
> >
> > > documented as "for experts".
> >
> > If a servlet must always subclass Servlet, or at least we pretend
> > to require that for the default case, the exception should be
> > changed from
> > AssertionError to something else.  AssertionError mean "I" made an
> > error.  Other exceptions mean "you" made an error.
> >
> > But perhaps we need to rethink what are the requirements for
> > servlets. At minimum, awake/respond/sleep methods.  This sounds
> > like an interface.
> > Since Python doesn't have interfaces (yet), we've been using
> > "must be a
> > subclass of Servlet" as a substitute.
> >
> > The simplest way to avoid the issubclass test and yet not introduce
> > an esoteric option like AllowServletsNotDerivedFromServlet would be
> > to test explicitly for the required features:
> >
> > 	if not hasattr(theClass, 'respond'): raise RuntimeError(...)
> >
> > 	try:
> > 		theClass.respond
> > 	except AttributeError:
> > 		raise RuntimeError(...)
> >
> > 	if not hasattr(theClass, 'respond') or \
> > 		type(theClass.respond) != types.MethodType:
> > 		raise RuntimeError(...)
> >
> > 	requiredMethods = 'awake', 'respond', 'sleep'
> > 	for methMame in requiredMethods:
> > 		try:
> > 			meth = getattr(theClass, methName)
> > 			if type(meth) != types.MethodType:
> > 				raise RuntimeError("not a method...")
> > 		except AttributeError:
> > 			raise RuntimeError("method doesn't exist...")
> >
> >
> > --
> > -Mike (Iron) Orr, iron@...  (if mail problems: mso@...)
> >    http://iron.cx/     English * Esperanto * Russkiy *
> > Deutsch * Espan~ol
> >
> > _______________________________________________
> > Webware-discuss mailing list
> > Webware-discuss@...
> > https://lists.sourceforge.net/lists/listinfo/webware-discuss

Thanks Chuck, all of your points seem valid to me.  But someone (I can't
even remember who started this thread :-)) is using a Delegation technique
to write his servlets and could only get it to work by removing the check
for Servlet as a base class.

Perhaps that someone could explain the technique in a bit more detail?  Why
are you using delegation instead of inheritance?  How exactly are you doing
it?

Maybe we can collectively find a way to accomplish your goals without having
to remove Servlet as a required base class.

- Geoff

> -----Original Message-----
> From: Chuck Esterbrook [mailto:ChuckEsterbrook@...]
> Sent: Wednesday, April 10, 2002 6:26 PM
> To: Geoffrey Talvola; webware-discuss@...
> Subject: Re: [Webware-discuss] Allowing delegation in servlets
> 
> 
> On Wednesday 10 April 2002 01:53 pm, Geoffrey Talvola wrote:
> > I would propose using "hasattr" to check for the existence of awake,
> > respond, and sleep, and removing the assert statement.  The
> > RuntimeError that results could have a much better error 
> message than
> > the assert would give us, anyhow.
> >
> > But I'm curious on Chuck's take on removing the assert statement.  I
> > somehow have a feeling that he has an opinion on this, maybe because
> > it was discussed before?
> >
> > - Geoff
> 
> Yes, this was discussed and decided before. Here are the points:
> 
> - By requiring subclasses of Servlet, we can put methods in Servlet 
> that other parts of WebKit (like the AppServer and Application) can 
> make use of.
> 
> Otherwise, if we decide to add some method like "serverSidePath" in a 
> version 0.4 then we basically break every WebKit application that 
> wasn't subclassing Servlet.
> 
> We would either get breakage every time we added something 
> (when there 
> is often a reasonable default implementation), or we could 
> have to make 
> these methods into global functions (that would presumably check 
> hasattr() first to see if the servlet wanted to "override them").
> 
> There is a "myth" among the Python community that requiring 
> inheritance 
> is _always_ bad. It is bad if you only rely on one or two methods and 
> don't plan on beefing up the de facto base class. It is also bad with 
> builtin types in Py <= 2.2 because they couldn't be subclassed.
> 
> - There are probably more required methods than you might think. I 
> doubt that it's only awake, respond and sleep.
> 
> - Since you are already using WebKit, inheriting from Servlet (or a 
> subclass is extremely easy):
> 
> from WebKit.HTTPServlet import HTTPServlet
> class Foo(HTTPServlet):
> 	pass
> 
> - Python provides multiple inheritance so you're not locked into just 
> inheriting from WebKit.
> 
> - Invoking hasattr() multiple times will be more expensive than one 
> isinstance().
> 
> - Checking isinstance() guards against the "duplicate module" bug we 
> experienced earlier.
> 
> 
> 
> It sounds like the assert should be changed to an "if: raise" to be 
> more clear and avoid being "optimized out".
> 
> -Chuck
> 
> 
> >
> > > -----Original Message-----
> > > From: iron@... [mailto:iron@...]
> > > Sent: Monday, April 08, 2002 3:31 PM
> > > To: webware-discuss@...
> > > Subject: Re: [Webware-discuss] Allowing delegation in servlets
> > >
> > > On Mon, Apr 08, 2002 at 02:43:37PM -0400, Geoffrey Talvola wrote:
> > > > > My first attempt failed because line 183 of ServletFactory is:
> > > > >
> > > > >     assert issubclass(theClass, Servlet)
> > > > >
> > > > > Anyway, I'd like to argue for the removal of the assertion in
> > > > > ServletFactory, since it is denying use of a very powerful,
> > > > > and arguably
> > > > > better technique than inheritance. In any case, assertions
> > > > > can be optimized
> > > > > away in .pyo files, so that the presence of the assertion
> > > > > statement is of
> > > > > questionable value...
> > > >
> > > > I think the assertion is there to help out the Webware user
> > >
> > > who forgets to
> > >
> > > > have their servlet inherit from Servlet.  I assume that
> > >
> > > mistake happens
> > >
> > > > relatively frequently.  I know I've done it once or twice.
> > >
> > > Without the
> > >
> > > > assertion (which pinpoints the problem pretty clearly),
> > >
> > > presumably you would
> > >
> > > > wind up with a traceback somewhere else in the guts of
> > >
> > > Appliction.py that
> > >
> > > > would be less obvious to track down, especially for a casual
> > > > user.
> > > >
> > > > Now you've identified a legitimate use for a servlet not
> > >
> > > derived from
> > >
> > > > Servlet.  So you want to remove the assertion, but that
> > >
> > > would hurt casual
> > >
> > > > Webware users hwo might forget the base class.  So maybe
> > >
> > > there should be an
> > >
> > > > option "AllowServletsNotDerivedFromServlet" that defaults
> > >
> > > to 0, but you can
> > >
> > > > just flip the switch and it skips the assertion?  I would
> > >
> > > imagine this to be
> > >
> > > > documented as "for experts".
> > >
> > > If a servlet must always subclass Servlet, or at least we pretend
> > > to require that for the default case, the exception should be
> > > changed from
> > > AssertionError to something else.  AssertionError mean "I" made an
> > > error.  Other exceptions mean "you" made an error.
> > >
> > > But perhaps we need to rethink what are the requirements for
> > > servlets. At minimum, awake/respond/sleep methods.  This sounds
> > > like an interface.
> > > Since Python doesn't have interfaces (yet), we've been using
> > > "must be a
> > > subclass of Servlet" as a substitute.
> > >
> > > The simplest way to avoid the issubclass test and yet not 
> introduce
> > > an esoteric option like 
> AllowServletsNotDerivedFromServlet would be
> > > to test explicitly for the required features:
> > >
> > > 	if not hasattr(theClass, 'respond'): raise RuntimeError(...)
> > >
> > > 	try:
> > > 		theClass.respond
> > > 	except AttributeError:
> > > 		raise RuntimeError(...)
> > >
> > > 	if not hasattr(theClass, 'respond') or \
> > > 		type(theClass.respond) != types.MethodType:
> > > 		raise RuntimeError(...)
> > >
> > > 	requiredMethods = 'awake', 'respond', 'sleep'
> > > 	for methMame in requiredMethods:
> > > 		try:
> > > 			meth = getattr(theClass, methName)
> > > 			if type(meth) != types.MethodType:
> > > 				raise RuntimeError("not a method...")
> > > 		except AttributeError:
> > > 			raise RuntimeError("method doesn't exist...")
> > >
> > >
> > > --
> > > -Mike (Iron) Orr, iron@...  (if mail problems: mso@...)
> > >    http://iron.cx/     English * Esperanto * Russkiy *
> > > Deutsch * Espan~ol
> > >
> > > _______________________________________________
> > > Webware-discuss mailing list
> > > Webware-discuss@...
> > > https://lists.sourceforge.net/lists/listinfo/webware-discuss
>

Chuck wrote:

>It sounds like the assert should be changed to an "if: raise" to be
>more clear and avoid being "optimized out".

Maybe so, but regardless of how the type checking is done, I think we ought
to be able to use delegation. With autodelegation, all the additional
methods that might be added to Servlet etc. in the future are still
available to unless specifically overridden by the delegating servlet.

Delegation is a superset of inheritance, and if it isn't allowed, we all
lose a rather important mechanism for managing all the issues that come
with just using inheritance to structure code.

I'm not advocating against inheritance. What I am advocating is that the
type check needs to be able to be turned off, or made a bit more
intelligent.

...Edmund.

On Wednesday 10 April 2002 04:00 pm, Edmund Lian wrote:
> Chuck wrote:
> >It sounds like the assert should be changed to an "if: raise" to be
> >more clear and avoid being "optimized out".
>
> Maybe so, but regardless of how the type checking is done, I think we
> ought to be able to use delegation. With autodelegation, all the
> additional methods that might be added to Servlet etc. in the future
> are still available to unless specifically overridden by the
> delegating servlet.
>
> Delegation is a superset of inheritance, and if it isn't allowed, we
> all lose a rather important mechanism for managing all the issues
> that come with just using inheritance to structure code.
>
> I'm not advocating against inheritance. What I am advocating is that
> the type check needs to be able to be turned off, or made a bit more
> intelligent.
>
> ...Edmund.

Can you give me a Python code example of the type of delegation you 
want to have and implement? It would help me wrap my head around the 
points you are raising.

Off the top of my head, it seems that you could do something with 
__getattr__ and/or __getattribute__ (new to Python 2.2), but we need to 
get more concrete on the nature and use of the delegation you desire.

(Sorry if you already did this and I missed the msg on the list. If 
that's the case, just tell me.)

-Chuck

Geoff wrote:

>But someone (I can't even remember who started this thread :-)) is
>using a Delegation technique to write his servlets and could only
>get it to work by removing the check for Servlet as a base class.

That someone was me... :-)

Here's an example of delegation:

In the file that I want Webware to treat as a servlet:

class PseudoServlet:
    def __init__(self):
        # Create and store instance of a real servlet
        self.__dict__['realservlet'] = Servlet()

    def myMethod1(self):
        do stuff

    def __getattr__(self, name):
        return getattr(self.realservlet, name)

    def __setattr__(self, name, value):
        return setattr(self.realservlet, name, value)

PseudoServlet can be served up by Webware as normal so long as the
issubclass() assertion in the ServletFactory is relaxed. What happens
is that any attribute or method that isn't defined in PseudoServlet
is automatically delegated to the real servlet.

Originally, I did this to get around a multiple inheritance problem that
I was having with Cheetah, where I could not override the base template's
.awake() method due to the way Python < 2.2 was resolving multiple
inheritance. Cheetah has since been modified to only support single
inheritance.

However, I'm still using delegation to get around weak encapsulation
and other issues that I ran into when writing servlets. The issues
I'm referring to are discussed in the following links:

1. Inheritance results in rigid interfaces and weak encapsulation of
attributes, methods, etc. of the superclasses, hence the need to be
aware of *all* the attributes and methods in superclasses when writing
a Cheetah template, new servlet, or mix-ins. There are other pros and
cons of the approaches, revolving around fragility of superclass
interfaces, etc. This is a good, short summary of some of these issues:
http://www.javaworld.com/javaworld/jw-11-1998/jw-11-techniques.html

2. The second problem I ran into was the generic multiple inheritance
problem that is recognized and addressed in Python 2.2. Kuchling
discusses this in
http://www.amk.ca/python/2.2/index.html#SECTION000330000000000000000" but
there are lots of references to this in the literature going back to
the early days of OO. Here's a good reference:
http://www.elj.com/eiffel/feature/inheritance/mi/review/

Problem 2 was interesting since the specific issue I had when Cheetah
still supported multiple inheritance was solved in Python 2.2. What this
means is that when you use multiple inheritance, the methods in base
classes
you think you are overriding can differ between Python < 2.2 and
Python >= 2.2. This is nasty! In the situation I came across, the code
would
have worked in Python 2.2, but would have broken in Python < 2.2.

...Edmund.

On Wed, 2002-04-10 at 17:05, Terrel Shumway wrote:
> HTTPRequest.adapterName() seems to be used frequently for building
> absolute URI's, but it does not work, for example, with
> 
> RewriteRule (.*) /cgi-bin/WebKit.cgi$1 [L,PT]
> 
> which maps /cgi-bin/WebKit.cgi/ to /
> 
> Am I missing something?

You, sir, are correct!  At least, I'm pretty sure you are.

I haven't tested this, or even tried it trivially, but maybe adding
something like:

[E=ORIG_URI:%{REQUEST_URI}] to the options of the rule will set a
ORIG_URI environmental variable that you can use...?  

  Ian

But if you are using ModRewrite in this way, an absolute
URI would not include the adaptername.

Luke

--- Terrel Shumway <tshumway@...> wrote:
> HTTPRequest.adapterName() seems to be used frequently for
> building
> absolute URI's, but it does not work, for example, with
> 
> RewriteRule (.*) /cgi-bin/WebKit.cgi$1 [L,PT]
> 
> which maps /cgi-bin/WebKit.cgi/ to /
> 
> Am I missing something?


=====
------------------
Reference Counting Garbage Collection:
Look out philosophy majors, things really DO
cease to exist when no one is looking at them!
------------------

__________________________________________________
Do You Yahoo!?
Yahoo! Tax Center - online filing with TurboTax
http://taxes.yahoo.com/

1) You need to setup a database connection pool
2) In the servlet grab a connection and use it like a python db object

This is all psudocode from memory so check it or correct it please

1)  create a module like configuration.py
from MiscUtils.DBPool import DBPool
from pyPgSQL import PgSQL
datapool = DBPool(PgSQL, 5, 'localhost::comsci:aaron:vector5')

#now you have a datapool object that is a pool of connections

2)  In the servlet:
from configuration import datapool

class servlet......

   def writeContent(self):
       conn=datapool.getConnection()
       c=conn.cursor()
       c.execute("Select * from table")
       rs=c.fetchall()
       for row in rs:
            self.writeln("<tr>")
            for field in row:
                self.writeln("<TD>%s</TD>" % field)
            self.writeln("</tr>")
       c.close()

-Aaron

> dennis@... wrote:
>> On Tue, 9 Apr 2002 dennis@... wrote:
>>
>> > I am writing my own database access at the moment. What
>> sorts of "gotchas"
>> > do I need to worry about? I am a little fuzzy as to how
>> WebKit works and
>> > what issues I face.
>>
>> Do I need to worry about threading issues, etc?
>
> Yes.  WebKit does use multiple threads, so you do have to worry about
> threading issues.  Any given servlet _instance_ is only run in a single
> thread at a time, which works around some, but not all, threading
> issues. But there will be multiple instances of a given servlet class
> running in separate threads.
>
> MiscUtils/DbPool.py provides a database connection pool that is
> designed to work safely in multiple threads.  You'll want to study it
> and possibly use it.
>
> - Geoff
>
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss

Tripp wrote:

>          Technically, UTC != GMT. See:
>
>          http://aa.usno.navy.mil/faq/docs/UT.html

Yes, but why complicate the issue even further?  :-)

Wow, how is it that you're only now reading and responding to stuff posted
in January???

...Edmund.

On Wed, 10 Apr 2002, Edmund Lian wrote:

> Yes, but why complicate the issue even further?  :-)

Hence the <pedantic/> tagset :)


> Wow, how is it that you're only now reading and responding to stuff posted
> in January???

Back in January, I was under a big deadline, so I only skimmed the list.
I've gotten a number of other big deadlines out of the way, so I'm taking
some time to catch up :)

I'm at least trying to read the entire thread (when it all occurs within a
few days) before replying with anything that might have already been said
:)