Home / Browse / Webware for Python / Mail / Archive

Webware for Python

Email Archive: webware-discuss (read-only)

RE: [Webware-discuss] rawInput does not work?

From: Geoffrey Talvola <gtalvola@na...> - 2003-04-29 15:25

Note: these comments apply to Webware 0.8.  I haven't looked at CVS
recently.
 
request.rawInput() only works if the request has an "irregular" content-type
such as text/xml which is what you get with XML-RPC, or
text/x-python-pickled-dict which is Pickle-RPC.  If the content-type is one
of the standard types multipart/form-data or
application/x-www-form-urlencoded, then the raw input is processed into
fields and discarded when the HTTPRequest object is initialized, presumably
for memory reasons.  This is all done in the Python standard library cgi
module, so it would be a bit of work for us to pry that apart and handle it
in a different way.
 
I have a suspicion that what you want to do is to reconstruct the raw input
in order to re-POST it to another HTTP server.  If so, then for
application/x-www-form-urlencoded data something like
 
    urllib.urlencode(self.request().fields(), doseq=1)
 
should work fine.
 
- Geoff
 

-----Original Message-----
From: Aaron Held [mailto:aaron@...]
Sent: Monday, April 28, 2003 3:37 PM
To: cprinos@...
Cc: jose@...; webware-discuss@...
Subject: Re: [Webware-discuss] rawInput does not work?


One of the problems seems to be that Webware just sucks up all of the raw
input.  Is there a way to recoved this?

I want to pull out the POST data directly, is there a good way?
The POST data is already in FieldStorage, but I want it just the way the
browser sent it.

Thanks,
-Aaron

Chris Prinos wrote:


-----Original Message-----

From:  webware-discuss-admin@...
<mailto:webware-discuss-admin@...> 

[ mailto:webware-discuss-admin@...
<mailto:webware-discuss-admin@...> ]On Behalf Of

jose@... <mailto:jose@...> 

Sent: Thursday, April 17, 2003 7:33 PM

    



  

I can't seem to get self.request().rawInput() (with or

without the rewind=1

option) to work.



Essentually I have a webpage that calls a servlet via a

form/Post and if I

then call self.writeln(self.request().rawInput()) I get back

None.  Now I

know the form is sending some data, because all it does is

send a hidden

variable.  Any thoughts as to what I am doing worng?

    



rawInput will only return a file if the data being POSTed can't be consumed

by the cgi/FieldStorage code first. If you're using a typical form posting,

then the content type would be application/x-www-form-urlencoded  and the

data from the form gets sent as encoded name value pairs. You can access

them from your servlet code using self.request().field(myFormFieldName). If

it's a multipart form, then you can get to the parts using

self.request().fieldStorage()



In either of those two cases there won't be a file available from

rawInput(), it's already been processed by that point. This shouldn't be a

problem though, as you should be able to access all the data through

self.request().field() or self.request().fieldStorage()



For content-types other than application/x-www-form-urlencoded or

multipart/form-data, you can use rawInput().



Chris







-------------------------------------------------------

This sf.net email is sponsored  by:ThinkGeek <by:ThinkGeek> 

Welcome to geek heaven.

http://thinkgeek.com/sf <http://thinkgeek.com/sf> 

_______________________________________________

Webware-discuss mailing list

Webware-discuss@...
<mailto:Webware-discuss@...> 

https://lists.sourceforge.net/lists/listinfo/webware-discuss
<https://lists.sourceforge.net/lists/listinfo/webware-discuss>

Thread View

Thread	Author	Date
RE: [Webware-discuss] rawInput does not work?	Geoffrey Talvola <gtalvola@na...>

RE: [Webware-discuss] rawInput does not work?

From: Terrel Shumway <tshumway-sf@jd...> - 2003-04-29 16:38

Quoting Geoffrey Talvola <gtalvola@...>:

> Note: these comments apply to Webware 0.8.  I haven't looked at CVS
> recently.
>  
> request.rawInput() only works if the request has an "irregular" content-type
> such as text/xml which is what you get with XML-RPC, or
> text/x-python-pickled-dict which is Pickle-RPC.  If the content-type is one
> of the standard types multipart/form-data or
> application/x-www-form-urlencoded, then the raw input is processed into
> fields and discarded when the HTTPRequest object is initialized, presumably
> for memory reasons.  

It doesn't seem to be discarded, and certainly not for memory reasons -- since
all of the data would still be held in the FieldStorage object.

request._input is the original file object.

when I try

	input = req._input
	input.seek(0)
	data = input.read()
I get

        IOError: [Errno 29] Illegal seek

which would make sense for OneShot, since _input is?? STDIN, but I get the same
result when I fire up AppServer.  hmmm.

The Java Servlet spec works around this by using lazy evaluation: request._input
is guaranteed to be available (untouched) until you access .fields()

Maybe Webware could do the same.


> This is all done in the Python standard library cgi
> module, so it would be a bit of work for us to pry that apart and handle it
> in a different way.

No, just delay calling cgi.FieldStorage() until you know it is needed.

I'm working on a patch if anyone is interested.

RE: [Webware-discuss] rawInput does not work?

From: Geoffrey Talvola <gtalvola@na...> - 2003-04-29 16:54

Terrel Shumway [mailto:tshumway-sf@...] wrote:
> Quoting Geoffrey Talvola <gtalvola@...>:
> 
> > Note: these comments apply to Webware 0.8.  I haven't looked at CVS
> > recently.
> >  
> > request.rawInput() only works if the request has an 
> > "irregular" content-type
> > such as text/xml which is what you get with XML-RPC, or
> > text/x-python-pickled-dict which is Pickle-RPC.  If the 
> > content-type is one
> > of the standard types multipart/form-data or
> > application/x-www-form-urlencoded, then the raw input is 
> > processed into
> > fields and discarded when the HTTPRequest object is 
> > initialized, presumably
> > for memory reasons.  
> 
> It doesn't seem to be discarded, and certainly not for memory 
> reasons -- since
> all of the data would still be held in the FieldStorage object.
> 
> request._input is the original file object.
> 
> when I try
> 
> 	input = req._input
> 	input.seek(0)
> 	data = input.read()
> I get
> 
>         IOError: [Errno 29] Illegal seek
> 
> which would make sense for OneShot, since _input is?? STDIN, 
> but I get the same
> result when I fire up AppServer.  hmmm.

It's because _input is really a file-like object made from a socket using
socket.makefile().  In ThreadedAppServer.py:

		dict['input'] = conn.makefile("rb",8012)

You can't seek on one of these pseudo-files since that would require it to
buffer up all input.

> 
> The Java Servlet spec works around this by using lazy 
> evaluation: request._input
> is guaranteed to be available (untouched) until you access .fields()
> 
> Maybe Webware could do the same.

Sounds like a good idea.

> 
> 
> > This is all done in the Python standard library cgi
> > module, so it would be a bit of work for us to pry that 
> > apart and handle it
> > in a different way.
> 
> No, just delay calling cgi.FieldStorage() until you know it is needed.
> 
> I'm working on a patch if anyone is interested.

Sure, go for it.

- Geoff

Re: [Webware-discuss] rawInput does not work?

From: Aaron Held <aaron@Me...> - 2003-04-29 17:21

>
>
>Terrel Shumway [mailto:tshumway-sf@...] wrote:
>  
>
>>No, just delay calling cgi.FieldStorage() until you know it is needed.
>>
>>I'm working on a patch if anyone is interested.
>>    
>>
If your messing with field storage it would also be useful to add 
additional information concerning where the data came from, URL/GET or POST
In order to do what I need I have to manually figure it our by parsing 
the query string and comparing it to fields().

On the other hand, this type of info is rarely needed - so maybe leave 
it out of the core handling.

-Aaron

RE: [Webware-discuss] rawInput does not work?

From: Terrel Shumway <tshumway-sf@jd...> - 2003-04-30 00:13

Quoting Geoffrey Talvola <gtalvola@...>:

> Terrel Shumway [mailto:tshumway-sf@...] wrote:
> > Quoting Geoffrey Talvola <gtalvola@...>:
> > 
> > The Java Servlet spec works around this by using lazy 
> > evaluation: request._input
> > is guaranteed to be available (untouched) until you access .fields()
> > 
> > Maybe Webware could do the same.
> 
> Sounds like a good idea.
> 
> > 
> > I'm working on a patch if anyone is interested.

It's currently working for me, but it will take a couple of days before I can
package it up neatly. Here is what I did (FYI/RFC):

1) move the field reading code from HTTPRequest.__init__ into a new method
readFields() 

2) add "if self._fields is None: self.readFields()" at the begining of each
method that accesses _fields.

3) removed the logic that accesses value("__captureStdOut__"), since it defeats
the purpose of the change, and is only useful for debugging.

4) made the session id logic reference cookies("_SID_") instead of
value("_SID_") (and some related changes). Instead of putting the _SID_ into a
Field, I added it directly as HTTPRequest._session_id (with a corresponding
._sid_source=[None|"path"|"cookie"]). 

This will break the feature that allows you to pass the _SID_ in the
query_string, but ../_SID_=..../ path rewriting is a better solution anyway
(IMO). Maybe the session/path code could also look directly in the query string
without triggering the POST reading code (if this feature is important enough to
preserve).

RE: [Webware-discuss] rawInput does not work?

From: Geoffrey Talvola <gtalvola@na...> - 2003-04-30 14:38

Terrel Shumway [mailto:tshumway-sf@...] wrote:
> 4) made the session id logic reference cookies("_SID_") instead of
> value("_SID_") (and some related changes). Instead of putting 
> the _SID_ into a
> Field, I added it directly as HTTPRequest._session_id (with a 
> corresponding
> ._sid_source=[None|"path"|"cookie"]). 
> 
> This will break the feature that allows you to pass the _SID_ in the
> query_string, but ../_SID_=..../ path rewriting is a better 
> solution anyway
> (IMO). Maybe the session/path code could also look directly 
> in the query string
> without triggering the POST reading code (if this feature is 
> important enough to
> preserve).

I used to use this "feature" of passing _SID_ in the query string, but I
don't any more.  I wouldn't object to removing it.

Another possible implementation would be for HTTPRequest to parse the query
string immediately and store in something like self._getFields.  Then the
session code could look there.  Of course, you would still lose the
"feature" of being able to POST the session ID as a _SID_ field, but I don't
know if anyone actually does that.

It also might be useful to remember which fields were GET fields and which
fields were POST fields and provide some interface for accessing them
separately.  I'm thinking of a "smart proxy" situation where you want to
gather up the GET and POST fields, perhaps modify or add certain values,
then use httplib to forward the modified request to another HTTP server.
You can't really do that correctly without being able to access the GET and
POST fields separately.

- Geoff

Re: [Webware-discuss] rawInput does not work?

From: Jeff Johnson <jeff@je...> - 2003-04-30 15:18

I pass the _SID_ on the query string for one of my sites that doesn't have an 
SSL cert for its domain.  Since I have an SSL cert for another generic 
payment domain on the same server, I just send the user to the secure domain 
with the _SID_, it then calls the same instance of webkit that was running on 
the unsecure domain.  If there is a different way of doing this, I'll switch, 
I just wanted you all to know that the current feature is being used.

-Jeff

On Wednesday 30 April 2003 10:38 am, Geoffrey Talvola wrote:
> Terrel Shumway [mailto:tshumway-sf@...] wrote:
> > 4) made the session id logic reference cookies("_SID_") instead of
> > value("_SID_") (and some related changes). Instead of putting
> > the _SID_ into a
> > Field, I added it directly as HTTPRequest._session_id (with a
> > corresponding
> > ._sid_source=[None|"path"|"cookie"]).
> >
> > This will break the feature that allows you to pass the _SID_ in the
> > query_string, but ../_SID_=..../ path rewriting is a better
> > solution anyway
> > (IMO). Maybe the session/path code could also look directly
> > in the query string
> > without triggering the POST reading code (if this feature is
> > important enough to
> > preserve).
>
> I used to use this "feature" of passing _SID_ in the query string, but I
> don't any more.  I wouldn't object to removing it.
>
> Another possible implementation would be for HTTPRequest to parse the query
> string immediately and store in something like self._getFields.  Then the
> session code could look there.  Of course, you would still lose the
> "feature" of being able to POST the session ID as a _SID_ field, but I
> don't know if anyone actually does that.
>
> It also might be useful to remember which fields were GET fields and which
> fields were POST fields and provide some interface for accessing them
> separately.  I'm thinking of a "smart proxy" situation where you want to
> gather up the GET and POST fields, perhaps modify or add certain values,
> then use httplib to forward the modified request to another HTTP server.
> You can't really do that correctly without being able to access the GET and
> POST fields separately.
>
> - Geoff
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> Welcome to geek heaven.
> http://thinkgeek.com/sf
> _______________________________________________
> Webware-discuss mailing list
> Webware-discuss@...
> https://lists.sourceforge.net/lists/listinfo/webware-discuss

Re: [Webware-discuss] rawInput does not work?

From: Aaron Held <aaron@Me...> - 2003-04-30 15:35

Attachments: Message as HTML

_SID_ on the query string is a very important feature if you need to 
serve client w/o cookies. I also don't use it, but as an appserver 
Webware should support 'cookie-less' sessions.

-Aaron Held

Jeff Johnson wrote:

>I pass the _SID_ on the query string for one of my sites that doesn't have an 
>SSL cert for its domain.  Since I have an SSL cert for another generic 
>payment domain on the same server, I just send the user to the secure domain 
>with the _SID_, it then calls the same instance of webkit that was running on 
>the unsecure domain.  If there is a different way of doing this, I'll switch, 
>I just wanted you all to know that the current feature is being used.
>
>-Jeff
>
>On Wednesday 30 April 2003 10:38 am, Geoffrey Talvola wrote:
>  
>
>>Terrel Shumway [mailto:tshumway-sf@...] wrote:
>>    
>>
>>>4) made the session id logic reference cookies("_SID_") instead of
>>>value("_SID_") (and some related changes). Instead of putting
>>>the _SID_ into a
>>>Field, I added it directly as HTTPRequest._session_id (with a
>>>corresponding
>>>._sid_source=[None|"path"|"cookie"]).
>>>
>>>This will break the feature that allows you to pass the _SID_ in the
>>>query_string, but ../_SID_=..../ path rewriting is a better
>>>solution anyway
>>>(IMO). Maybe the session/path code could also look directly
>>>in the query string
>>>without triggering the POST reading code (if this feature is
>>>important enough to
>>>preserve).
>>>      
>>>
>>I used to use this "feature" of passing _SID_ in the query string, but I
>>don't any more.  I wouldn't object to removing it.
>>
>>Another possible implementation would be for HTTPRequest to parse the query
>>string immediately and store in something like self._getFields.  Then the
>>session code could look there.  Of course, you would still lose the
>>"feature" of being able to POST the session ID as a _SID_ field, but I
>>don't know if anyone actually does that.
>>
>>It also might be useful to remember which fields were GET fields and which
>>fields were POST fields and provide some interface for accessing them
>>separately.  I'm thinking of a "smart proxy" situation where you want to
>>gather up the GET and POST fields, perhaps modify or add certain values,
>>then use httplib to forward the modified request to another HTTP server.
>>You can't really do that correctly without being able to access the GET and
>>POST fields separately.
>>
>>- Geoff
>>
>>
>>-------------------------------------------------------
>>This sf.net email is sponsored by:ThinkGeek
>>Welcome to geek heaven.
>>http://thinkgeek.com/sf
>>_______________________________________________
>>Webware-discuss mailing list
>>Webware-discuss@...
>>https://lists.sourceforge.net/lists/listinfo/webware-discuss
>>    
>>
>
>
>
>-------------------------------------------------------
>This sf.net email is sponsored by:ThinkGeek
>Welcome to geek heaven.
>http://thinkgeek.com/sf
>_______________________________________________
>Webware-discuss mailing list
>Webware-discuss@...
>https://lists.sourceforge.net/lists/listinfo/webware-discuss
>  
>