Webware for Python

It seems like there are three things going on here:

1. Customizing classes
2. Access privileges
3. Directory inheritance


I think 1. definitely goes into the existing 
WebKit/Configs/Application.config file. It would also be cool if WebKit 
would automatically pick up and use classes in applications that had the 
same name. e.g. you could just make a Session class that inherits from 
WebKit.Session and it would be used by convention of it's name. This 
wouldn't replace the configuration capability; it would just be a convenience.

Note that application does take arguments for the various classes and 
stores them as attributes. It then uses those when creating new sessions, 
requests, etc. So we're half way there. It's the config that's missing (as 
Dan has pointed out).


2. Access priv.: Not really sure how this should work. I certainly don't 
mind if someone takes a crack at it. Does this mean having classes like 
Role, User, etc.?

Or is using the authentication of the HTTP server possible and sufficient?


3. Directory inheritance: A Zope-/Quixote-inheritance scheme could be a 
useful thing, although there's obviously a performance hit for such things 
(e.g., scanning through each directory). That's why I prefer to use class 
inheritance rather than directory inheritance: It's all in memory and 
generally faster. But I can still see the utility of directory inheritance 
for easier configuration and better file system organization.

To get started, it might be useful to create a Page class/mix-in that 
offers methods such as:

	property()
	setProperty()
	object()
	setObject()

That are able to use files in directories under predetermined conventions. 
I'm picturing that properties are dictionary oriented, with the closest 
dictionaries being overlayed as you travel up the directory chain. In each 
directory there would be a Properties.config file. You would want to cache 
these and check the timestamps to have an efficient process while still 
having the ability to update them on disk at any time.

The objects would be larger things like graphics files or HTML fragments 
that are searched for up the directory chain and found according to 
filename. These could also be cached by time stamp.

You could also map extensions to Python classes, so that when an object is 
asked for, the class is instantiated with it and that's what you get.

For properties, you could special case dictionaries that have a key called 
'class'. For such dictionaries, the dictionary would be replaced by an 
instance of the class which would receive the readProperties() message.

These are just some brainstorming ideas on my part to warm up the conversation.


With regards to 2 & 3, what I'd really like to see are WebKit plug-ins 
rather than a new versions of WebKit.

The reasons are:
   * Plug-ins can be developed independently of each other.
   * People can then embrace as much or as little of the services we 
provide as what they want.
   * People can provide alternative versions of our services if they think 
they know how to do it better/faster.
   * Continuing with the plug-in architecture will encourage more plug-ins 
and Webware will evolve faster.

I'm more than willing to guide people in creating plug-ins. It's really 
just a small set of conventions for laying out your directories and files 
and making a function that lets you hook into WebKit at load time.

With regards to config files, I definitely prefer good ol' Python 
dictionaries rather than a custom format. Python config files allow for 
infinitely recursive dictionaries and lists and the only thing we have to 
do is eval().


Again, if people want to take a crack at plug-ins like this, I will 
certainly lend you a hand in ideas, testing, reviews, tech help on Webware, 
etc.

Provided something is produced that is useful, tested, documented and 
follows Webware conventions, we could put it in and you could aquire fame 
and fortune. Well, at least fame.  :-)

Thoughts?

-Chuck

Is the possibility of supporting multiple applications out of the question?
This would simplify a number of things.

For example, AppServer.config would now look something like:
{
    'PrintConfigAtStartUp': 1,
    'Verbose':              1,
    'Host':                 '127.0.0.1',
    'Port':                 8086,
    'PlugIns':              [],
    'PlugInDirs':           ['..'],
    'ServerThreads':        10,
    'Applications' : {
                        'MyApp' : 'MyApp.config',
                        'MyOtherApp' : 'MyOtherApp.config'
                     }
}

Application configuration stuff would be done in the specified files, rather
than just one configuration for the entire server. Alternatively, it could
even just be:

'Applications': [ 'MyApp', 'MyOtherApp' ]

and the names of the config files could be implied by the names of the
applications. We could take it one step further, and look for a class in the
Application's root folder named after that application, ie MyApp.py
containing a class called MyApp would be automatically loaded and used. If
it didn't exist, it would fall back on the generic Application class.

What would be the changes required to the internals of the WebKit to do
this? Things that spring to mind are:

- parsing the URI to determine which application should handle the request
- should the cache be per-Application, or for the entire server?
- some of the functionality in Application seems to be generic in that it
would be redundant to have multiple instances of Applications doing it
(error-checking URLs and so forth), so it would probably be best to move it
someplace else (ApplicationManager? ApplicationFactory?)
- would this obsolete the need for contexts? even if so, would it really be
worth removing them from the code? :)

I have yet to get a firm grasp on all the internals of the WebKit thus far,
so feel free to throw stuff at me and boo me off stage if I'm totally off
base. :)

As far as plug-ins go, I guess that since Python doesn't have private
members of classes, plug-ins pretty much have free reign to do what they
want, making them effectively unlimited in function. Am I right?

Dan Green

----- Original Message -----
From: "Chuck Esterbrook" <echuck@...>
To: "Daniel Green" <daniel.green@...>;
<webware-discuss@...>
Sent: Sunday, August 20, 2000 1:44 PM
Subject: Re: [Webware-discuss] Answer to a few problems?


> It seems like there are three things going on here:
>
> 1. Customizing classes
> 2. Access privileges
> 3. Directory inheritance
>
>
> I think 1. definitely goes into the existing
> WebKit/Configs/Application.config file. It would also be cool if WebKit
> would automatically pick up and use classes in applications that had the
> same name. e.g. you could just make a Session class that inherits from
> WebKit.Session and it would be used by convention of it's name. This
> wouldn't replace the configuration capability; it would just be a
convenience.
>
> Note that application does take arguments for the various classes and
> stores them as attributes. It then uses those when creating new sessions,
> requests, etc. So we're half way there. It's the config that's missing (as
> Dan has pointed out).
>
>
> 2. Access priv.: Not really sure how this should work. I certainly don't
> mind if someone takes a crack at it. Does this mean having classes like
> Role, User, etc.?
>
> Or is using the authentication of the HTTP server possible and sufficient?
>
>
> 3. Directory inheritance: A Zope-/Quixote-inheritance scheme could be a
> useful thing, although there's obviously a performance hit for such things
> (e.g., scanning through each directory). That's why I prefer to use class
> inheritance rather than directory inheritance: It's all in memory and
> generally faster. But I can still see the utility of directory inheritance
> for easier configuration and better file system organization.
>
> To get started, it might be useful to create a Page class/mix-in that
> offers methods such as:
>
> property()
> setProperty()
> object()
> setObject()
>
> That are able to use files in directories under predetermined conventions.
> I'm picturing that properties are dictionary oriented, with the closest
> dictionaries being overlayed as you travel up the directory chain. In each
> directory there would be a Properties.config file. You would want to cache
> these and check the timestamps to have an efficient process while still
> having the ability to update them on disk at any time.
>
> The objects would be larger things like graphics files or HTML fragments
> that are searched for up the directory chain and found according to
> filename. These could also be cached by time stamp.
>
> You could also map extensions to Python classes, so that when an object is
> asked for, the class is instantiated with it and that's what you get.
>
> For properties, you could special case dictionaries that have a key called
> 'class'. For such dictionaries, the dictionary would be replaced by an
> instance of the class which would receive the readProperties() message.
>
> These are just some brainstorming ideas on my part to warm up the
conversation.
>
>
> With regards to 2 & 3, what I'd really like to see are WebKit plug-ins
> rather than a new versions of WebKit.
>
> The reasons are:
>    * Plug-ins can be developed independently of each other.
>    * People can then embrace as much or as little of the services we
> provide as what they want.
>    * People can provide alternative versions of our services if they think
> they know how to do it better/faster.
>    * Continuing with the plug-in architecture will encourage more plug-ins
> and Webware will evolve faster.
>
> I'm more than willing to guide people in creating plug-ins. It's really
> just a small set of conventions for laying out your directories and files
> and making a function that lets you hook into WebKit at load time.
>
> With regards to config files, I definitely prefer good ol' Python
> dictionaries rather than a custom format. Python config files allow for
> infinitely recursive dictionaries and lists and the only thing we have to
> do is eval().
>
>
> Again, if people want to take a crack at plug-ins like this, I will
> certainly lend you a hand in ideas, testing, reviews, tech help on
Webware,
> etc.
>
> Provided something is produced that is useful, tested, documented and
> follows Webware conventions, we could put it in and you could aquire fame
> and fortune. Well, at least fame.  :-)
>
> Thoughts?
>
> -Chuck

At 10:39 PM 8/20/00 -0400, Daniel Green wrote:
>Is the possibility of supporting multiple applications out of the question?

Not at all. This was part of my original mental design, but something that 
I haven't gotten to yet. It's also the design used by WebObjects and a very 
appropriate one in my opinion. More below:


>This would simplify a number of things.
>
>For example, AppServer.config would now look something like:
>{
>     'PrintConfigAtStartUp': 1,
>     'Verbose':              1,
>     'Host':                 '127.0.0.1',
>     'Port':                 8086,
>     'PlugIns':              [],
>     'PlugInDirs':           ['..'],
>     'ServerThreads':        10,
>     'Applications' : {
>                         'MyApp' : 'MyApp.config',
>                         'MyOtherApp' : 'MyOtherApp.config'
>                      }
>}
>
>Application configuration stuff would be done in the specified files, rather
>than just one configuration for the entire server. Alternatively, it could
>even just be:
>
>'Applications': [ 'MyApp', 'MyOtherApp' ]

I would vote for 'Applications' being a dictionary that maps the name of 
the app to it's home directory. The name is what you would see in the URI. 
That's almost what contexts are now.


>and the names of the config files could be implied by the names of the
>applications. We could take it one step further, and look for a class in the

Yes, if/when we switch to this approach, we would grab "Application.config" 
out of the home dir of the app, overlay on the default WebKit 
Application.config and proceed.


>Application's root folder named after that application, ie MyApp.py
>containing a class called MyApp would be automatically loaded and used. If
>it didn't exist, it would fall back on the generic Application class.

My preference is to name the directory MyApp and then name the classes 
generically: Application.py, Session.py, etc. One advantage is that you 
only have to change the directory name to change the app name. Another is 
that you can copy (or link to) a file without having to modify it (although 
there are better ways to structure your reuse). And you'll also instantly 
recognize the classes when you examine someone else's app.


>What would be the changes required to the internals of the WebKit to do
>this? Things that spring to mind are:
>
>- parsing the URI to determine which application should handle the request

I would do that in the adapter which would then send the raw request to the 
application which would have it's own port.


>- should the cache be per-Application, or for the entire server?

per-Application (since I picture the app being a single process in it's own 
right).


>- some of the functionality in Application seems to be generic in that it
>would be redundant to have multiple instances of Applications doing it
>(error-checking URLs and so forth), so it would probably be best to move it
>someplace else (ApplicationManager? ApplicationFactory?)

Seems like these things have to be done per request that comes in. It's 
really the code that needs to be reused. I would leave it in application.


>- would this obsolete the need for contexts? even if so, would it really be
>worth removing them from the code? :)

Yes and if the new design was clean enough, then yes again.


>I have yet to get a firm grasp on all the internals of the WebKit thus far,
>so feel free to throw stuff at me and boo me off stage if I'm totally off
>base. :)

You're very much on base and obviously have a good handle on this stuff.


>As far as plug-ins go, I guess that since Python doesn't have private
>members of classes, plug-ins pretty much have free reign to do what they
>want, making them effectively unlimited in function. Am I right?

That is correct. You can add methods to classes, mix in base classes, etc.

However, all the stuff you described above would be WebKit specific, but 
then you're probably alluding to the other things we talked about.

Also, I think each application should be a separate process. After all, GUI 
apps don't share the same process space right? Let each application deal 
with it's own leaks, hang ups, deadlocks, port problems, etc.


-Chuck

> 
> I would vote for 'Applications' being a dictionary that maps the name of
> the app to it's home directory. The name is what you would see in the URI.
> That's almost what contexts are now.
> 
> >and the names of the config files could be implied by the names of the
> >applications. We could take it one step further, and look for a class in the
> 
> Yes, if/when we switch to this approach, we would grab "Application.config"
> out of the home dir of the app, overlay on the default WebKit
> Application.config and proceed.
> 
> >Application's root folder named after that application, ie MyApp.py
> >containing a class called MyApp would be automatically loaded and used. If
> >it didn't exist, it would fall back on the generic Application class.
> 
> My preference is to name the directory MyApp and then name the classes
> generically: Application.py, Session.py, etc. One advantage is that you
> only have to change the directory name to change the app name. Another is
> that you can copy (or link to) a file without having to modify it (although
> there are better ways to structure your reuse). And you'll also instantly
> recognize the classes when you examine someone else's app.


Perhaps a cleaner approach would be to define a directory hierarchy that
would be assumed.  So you would have the root directory of your
Context/Application(I'm still stuck on that term), and beneath it, you
might have at least three directories, Config (any config files),
Classes (Session, Application/Context, etc), Content or Servlets
(obvious).  More elaborate sites might have a Cans directory, a Static
directory, etc.




> 
> >What would be the changes required to the internals of the WebKit to do
> >this? Things that spring to mind are:
> >
> >- parsing the URI to determine which application should handle the request
> 
> I would do that in the adapter which would then send the raw request to the
> application which would have it's own port.


Then what's the difference from running multiple AppServers?  That's
essentially what you're saying here.  The only difference being you only
have one directory for WebKit.  In that case, we could just modify
AppServer to take a command line telling it which set of configuration
files to use.


> 
> >- should the cache be per-Application, or for the entire server?
> 
> per-Application (since I picture the app being a single process in it's own
> right).
> 
> >- some of the functionality in Application seems to be generic in that it
> >would be redundant to have multiple instances of Applications doing it
> >(error-checking URLs and so forth), so it would probably be best to move it
> >someplace else (ApplicationManager? ApplicationFactory?)
> 
> Seems like these things have to be done per request that comes in. It's
> really the code that needs to be reused. I would leave it in application.
> 
> >- would this obsolete the need for contexts? even if so, would it really be
> >worth removing them from the code? :)
> 
> Yes and if the new design was clean enough, then yes again.


I was intending to extend the Context concept to something like this. 
But Chuck and I have always differed on where to split the line between
AppServer, Application and Context.  (Well, I don't know that we differ
that much, we've just never reconciled our terminology ;) ) If you're
going to run on Multiple ports, I think you're splitting too high.

Jay

At 08:25 PM 8/21/00 -0400, Jay Love wrote:
>Perhaps a cleaner approach would be to define a directory hierarchy that
>would be assumed.  So you would have the root directory of your
>Context/Application(I'm still stuck on that term), and beneath it, you
>might have at least three directories, Config (any config files),
>Classes (Session, Application/Context, etc), Content or Servlets
>(obvious).  More elaborate sites might have a Cans directory, a Static
>directory, etc.

Sounds reasonably good.


>Then what's the difference from running multiple AppServers?  That's
>essentially what you're saying here.  The only difference being you only
>have one directory for WebKit.  In that case, we could just modify
>AppServer to take a command line telling it which set of configuration
>files to use.

It's not all that different, as you pointed out. But it would work out of 
the box and it would launch the application if needed.


>I was intending to extend the Context concept to something like this.
>But Chuck and I have always differed on where to split the line between
>AppServer, Application and Context.  (Well, I don't know that we differ
>that much, we've just never reconciled our terminology ;) ) If you're
>going to run on Multiple ports, I think you're splitting too high.

Not sure what you mean by splitting two high.


As a review, I'm thinking the future looks like this:

* Each app is it's own process
* Therefore, no two apps have the same process space or port number
* Apps can switch host:ports without warning (as a result of fault detection)
* Apps can have multiple host:ports (for fault tolerance and load balancing)
* Adapters should launch apps if needed
* Adapters should all inherit the abstract Adapter class
* AppServer remains and continues to play the role of network and 
multitasking management
* It's very easy to override the Application, Session, etc. classes through 
either a config file or naming conventions.
* We adopt some kind of directory structure (or lack thereof) and advertise it
* Contexts are subsumed by this new structure
* Everyone is happy.


:-)