Webware for Python

Quoting Geoffrey Talvola <gtalvola@...>:

> Note: these comments apply to Webware 0.8.  I haven't looked at CVS
> recently.
>  
> request.rawInput() only works if the request has an "irregular" content-type
> such as text/xml which is what you get with XML-RPC, or
> text/x-python-pickled-dict which is Pickle-RPC.  If the content-type is one
> of the standard types multipart/form-data or
> application/x-www-form-urlencoded, then the raw input is processed into
> fields and discarded when the HTTPRequest object is initialized, presumably
> for memory reasons.  

It doesn't seem to be discarded, and certainly not for memory reasons -- since
all of the data would still be held in the FieldStorage object.

request._input is the original file object.

when I try

	input = req._input
	input.seek(0)
	data = input.read()
I get

        IOError: [Errno 29] Illegal seek

which would make sense for OneShot, since _input is?? STDIN, but I get the same
result when I fire up AppServer.  hmmm.

The Java Servlet spec works around this by using lazy evaluation: request._input
is guaranteed to be available (untouched) until you access .fields()

Maybe Webware could do the same.


> This is all done in the Python standard library cgi
> module, so it would be a bit of work for us to pry that apart and handle it
> in a different way.

No, just delay calling cgi.FieldStorage() until you know it is needed.

I'm working on a patch if anyone is interested.

Terrel Shumway [mailto:tshumway-sf@...] wrote:
> Quoting Geoffrey Talvola <gtalvola@...>:
> 
> > Note: these comments apply to Webware 0.8.  I haven't looked at CVS
> > recently.
> >  
> > request.rawInput() only works if the request has an 
> > "irregular" content-type
> > such as text/xml which is what you get with XML-RPC, or
> > text/x-python-pickled-dict which is Pickle-RPC.  If the 
> > content-type is one
> > of the standard types multipart/form-data or
> > application/x-www-form-urlencoded, then the raw input is 
> > processed into
> > fields and discarded when the HTTPRequest object is 
> > initialized, presumably
> > for memory reasons.  
> 
> It doesn't seem to be discarded, and certainly not for memory 
> reasons -- since
> all of the data would still be held in the FieldStorage object.
> 
> request._input is the original file object.
> 
> when I try
> 
> 	input = req._input
> 	input.seek(0)
> 	data = input.read()
> I get
> 
>         IOError: [Errno 29] Illegal seek
> 
> which would make sense for OneShot, since _input is?? STDIN, 
> but I get the same
> result when I fire up AppServer.  hmmm.

It's because _input is really a file-like object made from a socket using
socket.makefile().  In ThreadedAppServer.py:

		dict['input'] = conn.makefile("rb",8012)

You can't seek on one of these pseudo-files since that would require it to
buffer up all input.

> 
> The Java Servlet spec works around this by using lazy 
> evaluation: request._input
> is guaranteed to be available (untouched) until you access .fields()
> 
> Maybe Webware could do the same.

Sounds like a good idea.

> 
> 
> > This is all done in the Python standard library cgi
> > module, so it would be a bit of work for us to pry that 
> > apart and handle it
> > in a different way.
> 
> No, just delay calling cgi.FieldStorage() until you know it is needed.
> 
> I'm working on a patch if anyone is interested.

Sure, go for it.

- Geoff

>
>
>Terrel Shumway [mailto:tshumway-sf@...] wrote:
>  
>
>>No, just delay calling cgi.FieldStorage() until you know it is needed.
>>
>>I'm working on a patch if anyone is interested.
>>    
>>
If your messing with field storage it would also be useful to add 
additional information concerning where the data came from, URL/GET or POST
In order to do what I need I have to manually figure it our by parsing 
the query string and comparing it to fields().

On the other hand, this type of info is rarely needed - so maybe leave 
it out of the core handling.

-Aaron

Quoting Geoffrey Talvola <gtalvola@...>:

> Terrel Shumway [mailto:tshumway-sf@...] wrote:
> > Quoting Geoffrey Talvola <gtalvola@...>:
> > 
> > The Java Servlet spec works around this by using lazy 
> > evaluation: request._input
> > is guaranteed to be available (untouched) until you access .fields()
> > 
> > Maybe Webware could do the same.
> 
> Sounds like a good idea.
> 
> > 
> > I'm working on a patch if anyone is interested.

It's currently working for me, but it will take a couple of days before I can
package it up neatly. Here is what I did (FYI/RFC):

1) move the field reading code from HTTPRequest.__init__ into a new method
readFields() 

2) add "if self._fields is None: self.readFields()" at the begining of each
method that accesses _fields.

3) removed the logic that accesses value("__captureStdOut__"), since it defeats
the purpose of the change, and is only useful for debugging.

4) made the session id logic reference cookies("_SID_") instead of
value("_SID_") (and some related changes). Instead of putting the _SID_ into a
Field, I added it directly as HTTPRequest._session_id (with a corresponding
._sid_source=[None|"path"|"cookie"]). 

This will break the feature that allows you to pass the _SID_ in the
query_string, but ../_SID_=..../ path rewriting is a better solution anyway
(IMO). Maybe the session/path code could also look directly in the query string
without triggering the POST reading code (if this feature is important enough to
preserve).

Terrel Shumway [mailto:tshumway-sf@...] wrote:
> 4) made the session id logic reference cookies("_SID_") instead of
> value("_SID_") (and some related changes). Instead of putting 
> the _SID_ into a
> Field, I added it directly as HTTPRequest._session_id (with a 
> corresponding
> ._sid_source=[None|"path"|"cookie"]). 
> 
> This will break the feature that allows you to pass the _SID_ in the
> query_string, but ../_SID_=..../ path rewriting is a better 
> solution anyway
> (IMO). Maybe the session/path code could also look directly 
> in the query string
> without triggering the POST reading code (if this feature is 
> important enough to
> preserve).

I used to use this "feature" of passing _SID_ in the query string, but I
don't any more.  I wouldn't object to removing it.

Another possible implementation would be for HTTPRequest to parse the query
string immediately and store in something like self._getFields.  Then the
session code could look there.  Of course, you would still lose the
"feature" of being able to POST the session ID as a _SID_ field, but I don't
know if anyone actually does that.

It also might be useful to remember which fields were GET fields and which
fields were POST fields and provide some interface for accessing them
separately.  I'm thinking of a "smart proxy" situation where you want to
gather up the GET and POST fields, perhaps modify or add certain values,
then use httplib to forward the modified request to another HTTP server.
You can't really do that correctly without being able to access the GET and
POST fields separately.

- Geoff

I pass the _SID_ on the query string for one of my sites that doesn't have an 
SSL cert for its domain.  Since I have an SSL cert for another generic 
payment domain on the same server, I just send the user to the secure domain 
with the _SID_, it then calls the same instance of webkit that was running on 
the unsecure domain.  If there is a different way of doing this, I'll switch, 
I just wanted you all to know that the current feature is being used.

-Jeff

On Wednesday 30 April 2003 10:38 am, Geoffrey Talvola wrote:
> Terrel Shumway [mailto:tshumway-sf@...] wrote:
> > 4) made the session id logic reference cookies("_SID_") instead of
> > value("_SID_") (and some related changes). Instead of putting
> > the _SID_ into a
> > Field, I added it directly as HTTPRequest._session_id (with a
> > corresponding
> > ._sid_source=[None|"path"|"cookie"]).
> >
> > This will break the feature that allows you to pass the _SID_ in the
> > query_string, but ../_SID_=..../ path rewriting is a better
> > solution anyway
> > (IMO). Maybe the session/path code could also look directly
> > in the query string
> > without triggering the POST reading code (if this feature is
> > important enough to
> > preserve).
>
> I used to use this "feature" of passing _SID_ in the query string, but I
> don't any more.  I wouldn't object to removing it.
>
> Another possible implementation would be for HTTPRequest to parse the query
> string immediately and store in something like self._getFields.  Then the
> session code could look there.  Of course, you would still lose the
> "feature" of being able to POST the session ID as a _SID_ field, but I
> don't know if anyone actually does that.
>
> It also might be useful to remember which fields were GET fields and which
> fields were POST fields and provide some interface for accessing them
> separately.  I'm thinking of a "smart proxy" situation where you want to
> gather up the GET and POST fields, perhaps modify or add certain values,
> then use httplib to forward the modified request to another HTTP server.
> You can't really do that correctly without being able to access the GET and
> POST fields separately.
>
> - Geoff
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss

_SID_ on the query string is a very important feature if you need to 
serve client w/o cookies. I also don't use it, but as an appserver 
Webware should support 'cookie-less' sessions.

-Aaron Held

Jeff Johnson wrote:

>I pass the _SID_ on the query string for one of my sites that doesn't have an 
>SSL cert for its domain.  Since I have an SSL cert for another generic 
>payment domain on the same server, I just send the user to the secure domain 
>with the _SID_, it then calls the same instance of webkit that was running on 
>the unsecure domain.  If there is a different way of doing this, I'll switch, 
>I just wanted you all to know that the current feature is being used.
>
>-Jeff
>
>On Wednesday 30 April 2003 10:38 am, Geoffrey Talvola wrote:
>  
>
>>Terrel Shumway [mailto:tshumway-sf@...] wrote:
>>    
>>
>>>4) made the session id logic reference cookies("_SID_") instead of
>>>value("_SID_") (and some related changes). Instead of putting
>>>the _SID_ into a
>>>Field, I added it directly as HTTPRequest._session_id (with a
>>>corresponding
>>>._sid_source=[None|"path"|"cookie"]).
>>>
>>>This will break the feature that allows you to pass the _SID_ in the
>>>query_string, but ../_SID_=..../ path rewriting is a better
>>>solution anyway
>>>(IMO). Maybe the session/path code could also look directly
>>>in the query string
>>>without triggering the POST reading code (if this feature is
>>>important enough to
>>>preserve).
>>>      
>>>
>>I used to use this "feature" of passing _SID_ in the query string, but I
>>don't any more.  I wouldn't object to removing it.
>>
>>Another possible implementation would be for HTTPRequest to parse the query
>>string immediately and store in something like self._getFields.  Then the
>>session code could look there.  Of course, you would still lose the
>>"feature" of being able to POST the session ID as a _SID_ field, but I
>>don't know if anyone actually does that.
>>
>>It also might be useful to remember which fields were GET fields and which
>>fields were POST fields and provide some interface for accessing them
>>separately.  I'm thinking of a "smart proxy" situation where you want to
>>gather up the GET and POST fields, perhaps modify or add certain values,
>>then use httplib to forward the modified request to another HTTP server.
>>You can't really do that correctly without being able to access the GET and
>>POST fields separately.
>>
>>- Geoff
>>
>>
>>-------------------------------------------------------
>>This sf.net email is sponsored by:ThinkGeek
>>Welcome to geek heaven.
>>http://thinkgeek.com/sf
>>_______________________________________________
>>Webware-discuss mailing list
>>Webware-discuss@...
>>https://lists.sourceforge.net/lists/listinfo/webware-discuss
>>    
>>
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>Webware-discuss mailing list
>Webware-discuss@...
>https://lists.sourceforge.net/lists/listinfo/webware-discuss
>  
>