Webware for Python

At 03:34 PM 5/28/2001 +0200, Manfred Nowak wrote:
>Hi all,
>
>i try to work with WebKit on my own Homepage.
>How do i start AppServer on the Server of my ISP
>or can i run WebKit only on Intranets ??
>
>Best regards
>
>Manfred

If you haven't yet run Apache and WebKit on your own machine, I suggest you 
download a copy of Apache and do so. It's quite convenient, even on Windows.

Regarding your ISP, start with OneShot.cgi so you can explore. The install 
guide has more details. Anything beyond that is something that you'll have 
to work out with your ISP. For example, if they provide FastCGI for Apache 
and allow your app server to run long periods of time, then you will be fine.

You should point out to them that the app server requires substantially 
less load on the server than CGI does.



-Chuck

On Mon, May 28, 2001 at 03:34:43PM +0200, Manfred Nowak wrote:
> i try to work with WebKit on my own Homepage.
> How do i start AppServer on the Server of my ISP
> or can i run WebKit only on Intranets ??

You'd need to find an ISP that will allow you to install and run
WebKit from your home directory, and is willing to make any
necessary modifications to their webserver configuration file.
There are not many ISPs at present willing to do this.  

However, there *are* Python-friendly and Zope-friendly ISPs,
so those would be the ones to approach first.  I don't know any
particular companies though since I just use my own servers.

Also, Webware is not yet multiuser-ready, meaning that if several
people (or virtual hosts) want to run Webware sites independently,
they must each install a separate copy of Webware.  This is fine for
a few sites but does not scale for an ISP that wants to host dozens
or hundreds of Webware sites.  However, there is work afoot to make
Webware multiuser eventually, so that it will install into a shared
directory and each user would have only their own configuration files.

-- 
-Mike (Iron) Orr, iron@...  (if mail problems: mso@...)
   http://iron.cx/     English * Esperanto * Russkiy * Deutsch * Espan~ol

Chuck Esterbrook <ChuckEsterbrook@...> wrote:
> You should point out to them that the app server requires substantially 
> less load on the server than CGI does.

Not to be a party-pooper or anything, but this isn't that true in this
context.  For the span of a single transaction Webware doesn't use as
much resources as a CGI with similar functionality.  But on a shared
host it really matters how much resources you use over time.  Webware
is still using resources when no one is using is transacting with it,
CGI does not.

A typical site constructed with CGI -- where there are a few portions
that are dynamic, and more portions that are static -- will probably
use less resources over time.  This is what most shared hosts are
targetting.  Also, a run-away process takes a lot of resources, and
CGI is better protected from this.

It would be spiffy if the adapters -- which are more isolated and
hopefully more robust -- could act as the superego for AppServer,
restarting it if it takes too long, doesn't respond, or isn't there at
all.  I suppose that could be as simple as calling an rc script as
[os.]system("webware restart") whenever the connection to the
AppServer is denied or times out.

BTW, I've had a hard time finding that rc script (that maybe Geoff
wrote?  Or someone else...)  It would be nice if it was part of
Webware, even if it was in a non-committal experimental directory, or
available as a seperate download somewhere.

  Ian

On 28 May 01, at 12:35, Ian Bicking wrote:

> BTW, I've had a hard time finding that rc script (that maybe
> Geoff wrote?  Or someone else...)  It would be nice if it was
> part of Webware, even if it was in a non-committal experimental
> directory, or available as a seperate download somewhere. 

You mean like an /etc/rc.d/init.d/webkit script that would work 
with RedHat (and similar systems)?  That would be cool (so would 
auto-restart functionality).

However, I finally got webkit started; perhaps a small addition to 
the install guide is in order.  I originally unpacked the webware 
tarball under /usr/local/webware but that didn't work.  It might be 
obvious to others (but not to me) that this was stupid, but it 
might help someone to mention that the webware tree should be 
underneath the apache html tree (or at least somewhere specified in 
an httpd.conf directive).  I also played with a WebwareDir=relative-
path but that didn't work either.

It looks like the requirements for webkit.cgi are as follows:

Either put the Webware source tree under an existing apache dir 
(such as /home/httpd/html/webware, /var/www/webware, or whatever) 
or make sure you add an httpd.conf directive such as:

<Directory /home/httpd/webware>
    Options [whatever you need]
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

Then use the full path to webware, enclosed in single quotes, in 
the WebKit.cgi file:

WebwareDir = '/home/httpd/html/webware'

Do the permissions stuff, and start the AppServer.

Is this pretty much correct?  Have I missed anything?  What are the 
security implications of WebKit wrt the apache permissions?  I'm 
still trying to grok the basics, so I haven't thought much about 
the security side (not that I could think much there anyway...)

Thanks in advance, Steve

Mike Orr wrote:

> On Mon, May 28, 2001 at 03:34:43PM +0200, Manfred Nowak wrote:
> > i try to work with WebKit on my own Homepage.
> > How do i start AppServer on the Server of my ISP
> > or can i run WebKit only on Intranets ??
>
> You'd need to find an ISP that will allow you to install and run
> WebKit from your home directory, and is willing to make any
> necessary modifications to their webserver configuration file.
> There are not many ISPs at present willing to do this.

What are these modifications?
   sorry, I'm a WebNewbie :-(

I have copied CGIWrapper , WebKit ,WebUtils and MiscUtils to my ISP.
CGIWrapper Examples works fine !

The only problem is to start the AppServer for WebKit.
Can i do that from within WebKit.cgi or  must my ISP start this?

am I the only one who want to use Webware  over an ISP ?


Manfred

On Monday 28 May 2001 12:15, Manfred Nowak wrote:
> The only problem is to start the AppServer for WebKit.
> Can i do that from within WebKit.cgi or  must my ISP
> start this?

I haven't worked with WebKit.cgi but I'm sure you could 
modify it to restart the AppServer if it's not running.  
Another option is to use a chron job to check up on the 
AppServer and restart it if necessary.
Tavis

Manfred Nowak <mn@...> wrote:
> > WebKit from your home directory, and is willing to make any
> > necessary modifications to their webserver configuration file.
> > There are not many ISPs at present willing to do this.
> 
> What are these modifications?
>    sorry, I'm a WebNewbie :-(

First prerequisite -- you have to have shell access to the ISP.

If you use the CGIAdapter they don't have to make any real
modifications to the webserver, except to get rid of the extra part of
the URL (like http://mydomain.com/cgi-bin/CGIAdapter.cgi/real/url =>
http://mydomain.com/real/url), which you can do with mod_rewrite.
Many hosts already have this installed, and if they don't they should,
because it's a generally useful tool.  With mod_rewrite you can put
stuff in your .htaccess file to control the URLs.

But the real problem is having the AppServer running at all times.  A
lot of hosts simply don't allow you to have applications run all the
time.  Maybe you can find one that does.

And then, well, I still haven't seen a good solution to making sure
the AppServer is always up, without manual intervention.  I'm
supposing you could do something like:

% nohup ./AppServer > /dev/null

That doesn't seem nearly robust enough for me.  But this should run
the AppServer in the background, and keep it running after you log
out.  Your ISP may very well kill the AppServer automatically, though,
or if they are lower tech simply get mad at you when they see it
has run for a few days.

> I have copied CGIWrapper , WebKit ,WebUtils and MiscUtils to my ISP.
> CGIWrapper Examples works fine !
> 
> The only problem is to start the AppServer for WebKit.
> Can i do that from within WebKit.cgi or  must my ISP start this?

You can't do it in WebKit.cgi, but you can do it like I said above.

> am I the only one who want to use Webware  over an ISP ?

I certainly would like to be able to do it, but I think I've resolved
that I'll need to go with a virtual private server.  Someone suggested
several of these some months ago, but a search on geocrawler gives me
nothing.  Which is always the case -- making me believe that
geocrawler is rather defective and generally useless.

But, I digress.  One example of a VPS host is at:
  http://services.superb.net/sps/

Otherwise it is really necessary to find someone who is willing
specifically to host Webware, probably at a higher rate.  This is
pretty much the case if you want to run Zope, which is a fairly
similar situation.

If you find someone, please tell -- the website should list them.

  Ian

* Ian Bicking <ianb@...> [010528 23:59]:
> 
> Otherwise it is really necessary to find someone who is willing
> specifically to host Webware, probably at a higher rate.  This is
> pretty much the case if you want to run Zope, which is a fairly
> similar situation.
> 
> If you find someone, please tell -- the website should list them.

I know that my host is friendly enough.
He has Zope installed and will give access if you request it, you get
shell account and the ability to run processes of your own there
(assuming you don't overload the machine), they will even install things
that you need if you request nicely and they won't interfere with the
rest of their users. They've installed for me Python and it's database
interface for PostgreSQL, pretty accomodating.

http://www.synecdoche.net/

It's 20$ a month.

I'm not affiliated with them, just a happy customer.

-- 
Baruch Even
http://baruch.ev-en.org/

At 12:35 PM 5/28/2001 -0500, Ian Bicking wrote:
>Chuck Esterbrook <ChuckEsterbrook@...> wrote:
> > You should point out to them that the app server requires substantially
> > less load on the server than CGI does.
>
>Not to be a party-pooper or anything, but this isn't that true in this
>context.  For the span of a single transaction Webware doesn't use as
>much resources as a CGI with similar functionality.  But on a shared
>host it really matters how much resources you use over time.  Webware
>is still using resources when no one is using is transacting with it,
>CGI does not.

When Webware is not being used, it would presumably get swapped out to disk 
by the O.S. not given much priority. As it gets used, it would be swapped 
back in, which seems no worse than launching a CGI.

Consider that for *every request* CGI will incur this additional overhead:

* Launch a process
* Load the interpreter (python)
* Load the standard modules (os, sys, etc.)
* Load commonly used modules (MySQLdb, DateTime, etc.)
* Load the script (MyPage.py)
* Other initialization (e.g., opening a database connection)
* Shutdown the process

How many requests would it take for Webware to show a worthwhile gain over 
CGI? I'm guessing not too many.


>A typical site constructed with CGI -- where there are a few portions
>that are dynamic, and more portions that are static -- will probably
>use less resources over time.  This is what most shared hosts are

Not sure what you are implying about the "more portions are static". Even 
if the CGI spews forth HTML pieces that are fairly static, all the overhead 
I list above still takes place. The only static pieces that avoid that are 
things you like to like images, stylesheets and JavaScript. Which in both 
cases should be cached by the browser.


>targetting.  Also, a run-away process takes a lot of resources, and
>CGI is better protected from this.

True.


>BTW, I've had a hard time finding that rc script (that maybe Geoff
>wrote?  Or someone else...)  It would be nice if it was part of
>Webware, even if it was in a non-committal experimental directory, or
>available as a seperate download somewhere.

It's in CVS as WebKit/webkit.  Sheesh, isn't that obvious?

I think it's missing some contribs which I have on my TO DO list. Although 
I *am* using it on 2 projects and it seems to work just fine.

It's been on my list to give it a better name and possibly put it somewhere 
else. Maybe WebKit/Aux/webkit_init


-Chuck

At 12:28 PM 5/28/2001 -0700, Stephen L Arnold wrote:
>Either put the Webware source tree under an existing apache dir
>(such as /home/httpd/html/webware, /var/www/webware, or whatever)
>or make sure you add an httpd.conf directive such as:
>
><Directory /home/httpd/webware>
>     Options [whatever you need]
>     AllowOverride None
>     Order allow,deny
>     Allow from all
></Directory>

I don't really follow you at all. I have *never* put Webware under an 
Apache directory. Although it should work from anywhere both inside and out 
of Apache.

What I did have to do, was tweak WebKit.cgi as described in the Install 
Guide (primarily to point to Webware) and make sure I could execute CGIs. I 
guess also make sure that Webware was readable by the uid that would exec 
the CGI.


When using mod_webkit, I had to add this:

# webkit.conf
# This is a sample httpd.conf section.  I just include this file in
# my main httpd.conf with an Include directive.
# ie. Include /Webware/WebKit/mod_webkit/apache.conf


LoadModule webkit_module modules/mod_webkit.dll
AddModule mod_webkit.c

AddType text/psp .psp
AddHandler psp-handler .psp

<Location /webkit>
WKServer localhost 8086
SetHandler webkit-handler
</Location>



-Chuck

On Tue, May 29, 2001 at 07:26:10AM -0400, Chuck Esterbrook wrote:
> It's been on my list to give it a better name and possibly put it somewhere 
> else. Maybe WebKit/Aux/webkit_init

"aux" is a device name in Windows (at least 95/98), so any tarball
containing it won't unpack.

-- 
-Mike (Iron) Orr, iron@...  (if mail problems: mso@...)
   http://iron.cx/     English * Esperanto * Russkiy * Deutsch * Espan~ol

At 07:08 AM 5/29/2001 -0700, Mike Orr wrote:
>On Tue, May 29, 2001 at 07:26:10AM -0400, Chuck Esterbrook wrote:
> > It's been on my list to give it a better name and possibly put it 
> somewhere
> > else. Maybe WebKit/Aux/webkit_init
>
>"aux" is a device name in Windows (at least 95/98), so any tarball
>containing it won't unpack.

Hahaha.


Woops.


How about?
   Auxiliary
   Accessories
   Additions

I prefer the first.


-Chuck

Chuck Esterbrook <ChuckEsterbrook@...> wrote:
> When Webware is not being used, it would presumably get swapped out to disk 
> by the O.S. not given much priority. As it gets used, it would be swapped 
> back in, which seems no worse than launching a CGI.

That's a good point.

> Consider that for *every request* CGI will incur this additional overhead:
> 
> * Launch a process
> * Load the interpreter (python)
> * Load the standard modules (os, sys, etc.)
> * Load commonly used modules (MySQLdb, DateTime, etc.)
> * Load the script (MyPage.py)
> * Other initialization (e.g., opening a database connection)
> * Shutdown the process
> 
> How many requests would it take for Webware to show a worthwhile gain over 
> CGI? I'm guessing not too many.

Well, that's if you are doing Python CGI for every page.  This is
poorly supported on most hosts anyway.  Typically people either have a
small set of Python (or usually Perl) CGI scripts, or else they have a
bunch of PHP or ASP files.  If you have a database driven site, you
would probably do best to use PHP or ASP.

> >A typical site constructed with CGI -- where there are a few portions
> >that are dynamic, and more portions that are static -- will probably
> >use less resources over time.  This is what most shared hosts are
> 
> Not sure what you are implying about the "more portions are static". Even 
> if the CGI spews forth HTML pieces that are fairly static, all the overhead 
> I list above still takes place. The only static pieces that avoid that are 
> things you like to like images, stylesheets and JavaScript. Which in both 
> cases should be cached by the browser.

Many CGI-based websites only have CGI pages for specific parts of the
page.  They may generate the pages on a schedule, but just serve .html
files.

This is what hosts seem to encourage.  The cgi-bin directory certainly
is meant to support this sort of use.  Though there are a number of
websites who are nearly entirely served off of cgi-bin.

I'm under the impression that PHP is already persistent in many ways.

  Ian

* Ian Bicking <ianb@...> [010529 22:06]:
> 
> I'm under the impression that PHP is already persistent in many ways.
> 

PHP by itself is not persistent at all, PHP3 can't persist anything and
PHP4 can with appropriate add-ons, specifically PHP4 can feature an
optimizer and a cache, where the optimizer optimizes the PHP code and
the cache keeps the optimized P-code in store. There is no persistency
of data, unless you program it in the php code itself.

-- 
Baruch Even
http://baruch.ev-en.org/

Baruch Even <baruch@...> wrote:
> PHP by itself is not persistent at all, PHP3 can't persist anything and
> PHP4 can with appropriate add-ons, specifically PHP4 can feature an
> optimizer and a cache, where the optimizer optimizes the PHP code and
> the cache keeps the optimized P-code in store. There is no persistency
> of data, unless you program it in the php code itself.

The interpreter is persistent in mod_php, though, isn't it?  This
means an overhead to read and compile the page and user libraries, but
that's considerably less overhead (at least compared to loading the
interpreter and the code that is standard in PHP).

I think Zend's products do caching of user code, but that's a
commercial product.

  Ian

* Ian Bicking <ianb@...> [010529 22:28]:
> Baruch Even <baruch@...> wrote:
> > PHP by itself is not persistent at all, PHP3 can't persist anything and
> > PHP4 can with appropriate add-ons, specifically PHP4 can feature an
> > optimizer and a cache, where the optimizer optimizes the PHP code and
> > the cache keeps the optimized P-code in store. There is no persistency
> > of data, unless you program it in the php code itself.
> 
> The interpreter is persistent in mod_php, though, isn't it?  This
> means an overhead to read and compile the page and user libraries, but
> that's considerably less overhead (at least compared to loading the
> interpreter and the code that is standard in PHP).

The interpreter itself is either a module (mod_php) or a cgi, as a
module it is persistent as a cgi it's a compiled C program that gets
loaded.

> I think Zend's products do caching of user code, but that's a
> commercial product.

There are free cache modules for PHP4, I believe there is an ongoing
work to create a free optimizer for PHP4.

-- 
Baruch Even
http://baruch.ev-en.org/