Bugs item #1582744, was opened at 2006-10-23 09:44
Message generated for change (Comment added) made by cheengshuchin
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=551954&aid=1582744&group_id=78018
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: win32
Group: None
Status: Closed
Resolution: Invalid
Priority: 5
Private: No
Submitted By: cheeng shu chin (cheengshuchin)
Assigned to: Nobody/Anonymous (nobody)
Summary: isapi extension seem Error for large file posting in iis
Initial Comment:
hi all,
I try to submit a large file (2Mb+) to iis with
pywin32 isapi extension via html form (POST method),
but end out with "The page cannot be displayed" for IE
and "server connection reset" for firefox.
This happen even with attached module "C:\Python24
\Lib\site-packages\isapi\test\extension_simple.py".
The form with this "CONTENT_TYPE=multipart/form-data;
boundary=---------------------------7d629bf20426"
is it a limitation in isapi or some bug in it???
can someone help me on this???
----------------------------------------------------------------------
>Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-24 07:11
Message:
Logged In: YES
user_id=1546069
yes..fully agree with you....
sorry for it...as i post it as a bug priviously...
thank a lot for your help...:)
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2006-10-24 06:45
Message:
Logged In: YES
user_id=14198
I'm closing this as I don't believe it relates to a bug in
pywin32. If you just need general help with ISAPI, the
pywin32 bugs isn't the place; the pyt...@py...
mailing list would be more appropriate.
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2006-10-24 06:43
Message:
Logged In: YES
user_id=14198
You don't want to read all the data in one chunk - you may
exhaust memory if the file is truly huge. A better solution
is to loop reading a fixed size each time (eg, 8k)
These semantics are imposed by IIS and ISAPI. I think it is
best to expose these semantics directly to the programmer,
rather than trying to outsmart their buffering and other
performance related facilities. This also lets you consult
the IIS documentation for the exact semantics, something you
should do as it will answer other questions you have about
how IIS manages the input stream.
Cheers
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-24 05:43
Message:
Logged In: YES
user_id=1546069
I guess i know what was happened and solved the puzzle...:)(Please do correct me if i'm wrong)
found:
ecb.ReadClient is a block function same as file object read method.
all POST data in iis buffer with total size = ecb.TotalBytes
ecb always try to read those data from buffer as many as possible with return read size in ecb.AvailableBytes.
when ecb.TotalBytes<>ecb.AvailableBytes:
it mean still have some data leave in the iis buffer (specially for a large file, which bigger then ecb buffer store)
if we directly close the ecb with ecb.DoneWithSession()..it will trigger iis to raise error,
as you didn't finish to read it out from its own buufer.
this will follow by return error page to client browser with "The page cannot be displayed" ....:(
as long as you extract all data from IIS buffer, the web site will work correctly.....:)
so the solution:
=======================================================
def test2(self, ecb):
print 'TotalBytes :',ecb.TotalBytes
print 'AvailableBytes :',ecb.AvailableBytes
print 'Method :',ecb.Method
print 'ContentType :',ecb.ContentType
#print 'AvailableData :',ecb.AvailableData
st=ecb.AvailableData
if ecb.TotalBytes<>ecb.AvailableBytes:
print ecb.TotalBytes,ecb.AvailableBytes
st+=ecb.ReadClient(ecb.TotalBytes-ecb.AvailableBytes)
print '\r\n##_______________________________________________________##\r\n'
print st
print '\r\n##_______________________________________________________##\r\n'
return 'csc'
=======================================================
then it seem work fine now..........the page return with correct respond and everybody happy with it......:)
Note:
1. i still need to check the data after POST is identicall to original source.
2. i still need to convert isapi vars to CGI enviroment vars
3. i still need to represent those data in XML form, which will have greater potential to any kind of web base subsystem
suggestion:
1 can ecb just return all complete/full data in buffer ruther then call ecb.ReadClient repeatly???
2 or just let ecb collect all others data except ecb.AvailableData, which still leave those data in iis buffer and we just
make a call ecb.ReadClient(ecb.TotalBytes) to read it out?????
special thank to mark (one of my respected programmer) for yours valuable advises and support.
thank a lot.....
Mark, Please do correct me if i'm wrong....
BTW i will do further test to confirm the case.....
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2006-10-24 05:25
Message:
Logged In: YES
user_id=14198
Yes, as the ISAPI docs state, .AvailableData is only the
available data, and ReadClient() must be used to read the rest.
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-24 04:47
Message:
Logged In: YES
user_id=1546069
it seem like isapi ecb can return/extract all uploaded data from iis..and still lift some in the buffer.
should i use ecb.ReadClient method in a loop to extract all data in buffer????
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-24 04:40
Message:
Logged In: YES
user_id=1546069
i had check with 'live headers' seem no error.
is it a memory leak issue in here???
as i still can post a large file(python-2.4.4.msi) to the page(>2MB).
I still have result from win32traceutil window output but seem "truncated"....
and browser reply with "The page cannot be displayed"
please refer to attachment.....:)
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-24 04:28
Message:
Logged In: YES
user_id=1546069
if I post a smaller file (<1MB), the semple seem work fine, but will fail on large file....
i found it('live headers') already, as priviously i try to find it via FF extension search only...:(
those sample(extension_simple.py & advanced.py) only work with small file submit but not for bigggg file....
FYI:
====================
<HTML>
<HEAD>
<TITLE>Post Data</TITLE>
<META NAME="generator" CONTENT="ToniArts
EasyHtml v.2.2">
</HEAD>
<BODY>
<FORM method="POST"
action="http://localhost/pyisapi_test/test2?debug=1"
enctype='multipart/form-data'>
<TEXTAREA name=csc[1] cols=35 rows=7></TEXTAREA>
<TEXTAREA name=csc[1] cols=35 rows=7></TEXTAREA>
<TEXTAREA name=csc[2] cols=35 rows=7></TEXTAREA>
<TEXTAREA name=csc[3] cols=35 rows=7></TEXTAREA>
<input type="file" name="filen0" >
<input type="file" name="filen1" >
<INPUT type="Submit" value="Submit">
</FORM>
</BODY>
</HTML>
====================
new function define in extension_simple.py
-----------------------
def test2(self, ecb):
print 'TotalBytes :',ecb.TotalBytes
print 'AvailableBytes :',ecb.AvailableBytes
print 'Method :',ecb.Method
print 'ContentType :',ecb.ContentType
print 'AvailableData :',ecb.AvailableData
return 'csc'
---------------------
please check the output in attachment....:)
But this fail if you select a big file(>2MB)......
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2006-10-23 12:57
Message:
Logged In: YES
user_id=14198
Yep - in Aus - nearly bed time :) I suggest we try and get
some of the samples working first. Let's just take this to
email, but unless the samples work, your POST example isn't
going to.
I typed "live headers" into google and the first hit was the
correct link from mozdev.org, as were most of the top 10 -
I'm surprised you had trouble finding it.
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-23 11:07
Message:
Logged In: YES
user_id=1546069
hi mark,
i can't find "live headers" extension for FF.
so i can't provide you the header info....:(
BTW thank for your help...
r u still in aust.?? (almost 7:30pm now...:( )
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-23 10:52
Message:
Logged In: YES
user_id=1546069
nope...non of them(I'm afraid)....
actually we can POST file to iis server
via "enctype='multipart/form-da". and we can read it out
via isapi ecb.AvailableData, ecb.ContentType and with total
bytes size ecb.TotalBytes.
then we can use cgi module parse it after inject some isapi
vars to os.enviroment(to make it become seem like CGI).
Then I can expost all info in XML form via 4suit....:)
I try to find "live headers" and try it again....:)
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2006-10-23 10:37
Message:
Logged In: YES
user_id=14198
I'm afraid I don't know how ISAPI is supposed to handle post
data. You said you can't get the 'advanced' sample working
- do any of them work? Maybe try the 'live headers' firefox
extension - it may give you more clues what is going on.
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-23 10:26
Message:
Logged In: YES
user_id=1546069
I try to dig further, found that ECB seem work and recieve
post data, even i can redirect those data to win32traceutil
window include http vars or "print >>ecb, data", but it
always end out with "The page cannot be displayed" for IE
and "server connection reset" for firefox.
......???????!!!!!....:)
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-23 10:13
Message:
Logged In: YES
user_id=1546069
hi mark,
thank for your fast respond.....:)
nothing in win32traceutil window(except "Tester dispatching
******").
i had try "advanced.py" as well, seem not work....
Attached with html form
=====================
<HTML>
<HEAD>
<TITLE>Post Data</TITLE>
<META NAME="generator" CONTENT="ToniArts EasyHtml v.2.2">
</HEAD>
<BODY>
<FORM method="POST"
action="http://localhost/pyisapi_test/test1?debug=1"
enctype='multipart/form-data'>
<TEXTAREA name=csc[1] cols=35 rows=7></TEXTAREA>
<TEXTAREA name=csc[1] cols=35 rows=7></TEXTAREA>
<TEXTAREA name=csc[2] cols=35 rows=7></TEXTAREA>
<TEXTAREA name=csc[3] cols=35 rows=7></TEXTAREA>
<input type="file" name="filen0" >
<input type="file" name="filen1" >
<INPUT type="Submit" value="Submit">
</FORM>
</BODY>
</HTML>
===============================
BTW, Thank a lot for your help...:)
----------------------------------------------------------------------
Comment By: Mark Hammond (mhammond)
Date: 2006-10-23 10:06
Message:
Logged In: YES
user_id=14198
Do you see a traceback in the win32traceutil window?
Otherwise you will need to provide a sample which
demonstrates the problem
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-23 09:54
Message:
Logged In: YES
user_id=1546069
some correction....:
I can't compile it to any kind of isapi DLL as pywin32
ISAPI do...:(
sorry for that...:(
----------------------------------------------------------------------
Comment By: cheeng shu chin (cheengshuchin)
Date: 2006-10-23 09:52
Message:
Logged In: YES
user_id=1546069
i did samething with pyisapie mudule, it seem work. but i
can compile it to any kind of dll.
can i know what was happend in isapi module???
BTW, Thank a lot for those look in this.....:)
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=551954&aid=1582744&group_id=78018
|