From: Peter S. <pet...@in...> - 2003-09-29 22:16:56
|
Hello, I have a problem that I actually don't think is specific to webware, but which occurrs with it (and Spyce). In any case, I suspect I am not the only one to have run into it. I was about to write a short version followed by a long version, but the short version ended up being long, so nevermind: I have a servlet that pushes large amounts of data to the client (file download). This is done inside of a loop. The problem is that sometimes (or always, depending on browser I think - not fully investigated, see below for details) when the client closes the tcp connection prematurely, the python process/thread runniung the servlet starts looping with 100% cpu usage. I did a strace on the process and it seemed to be doing this repeatedly: send(9, "3\3457_~\250N\210hw\377.-)\1M\233C$7{\235B\5\3074[\34&"..., 8192, 0) = -1 EPIPE (Broken pipe) send(9, "3\3457_~\250N\210hw\377.-)\1M\233C$7{\235B\5\3074[\34&"..., 8192, 0) = -1 EPIPE (Broken pipe) And the loop in my code gets run several time per second (though not more than perhaps 25/second, so each write() seems to take some time). Now the question is: (1) Why isn't an exception raised? (2) write() returns None, so a theory someone had on #python @ freenode that I should be checking the return code, and not assuming full write even though it's blocking I/O, was not applicable. (3) How am I supposed to handle this? Note that this only happens under some circumstances. When I do it with links locally on the box (aborting a transfer mid-way) it always happens. With Opera over a 1 mbit link, it seems to happen sometimes, but not always (which would explain why I didn't catch it in testing). With MSIE it seems to happen at least sometimes, because this whole thing started when the system was deployed and I had 50 people downloading at the same time, most of them using MSIE. It may also be dependent on some state on the server. During initial observations it seemed that everything worked fine for quite a while (with 50 people downloading and several requests/second for other pages being handled at the same time), and then after some random point in time the load quickly spiked from around 1 to 40+ (over 100 after a while). This however was with spyce; the little testing I have done so far after porting the thing to WebWare indicates it's more immediate/common with WebWare (not meaning WebWare is doing something wrong; just a co-relation). I am running the app through apache->wkcgi->webware. The try/finally block with the while loop in my code looks like this (ignore the downloadManager stuff, it's just book keeping using poorly named methods...): try: file = open(filePath, 'r') stat = os.stat(filePath) response.setHeader('Content-Type', 'application/octet-stream') response.setHeader('Content-Length', str(stat.st_size)) response.setHeader('Content-Disposition', 'attachment; filename=' + filename) while 1: downloadManager.registerDownloader(verifier) data = file.read(50000) if len(data) == 0: break bytesWritten = response.write(data) response.flush() downloadManager.registerBytesDownloaded(len(data)) finally: downloadManager.unregisterDownloader(verifier) if file: file.close() -- / Peter Schuller, InfiDyne Technologies HB PGP userID: 0xE9758B7D or 'Peter Schuller <pet...@in...>' Key retrival: Send an E-Mail to get...@sc... E-Mail: pet...@in... Web: http://www.scode.org |
From: Peter S. <pet...@in...> - 2003-09-30 21:18:03
|
> for details) when the client closes the tcp connection prematurely, the > python process/thread runniung the servlet starts looping with 100% cpu > usage. Ok, I have confirmed that the problem occurrs with mod_webkit aswell. Also, I have been trying to understand what might be going on. But I cannot for the life of me find the location in the code where the data actually gets sent to the client. HTTPResponse.write() does write the data to _strmOut, but that is a TASASStreamOut according to the debug output I put in before the write(). The TASASStreamOut class only puts data in an in-memory buffer (even when flushing) and I cannot locate where that data is actually pulled out of the stream and sent to the socket. In either case, it is clear that def createResponseInTransaction(self, transaction, strmOut): response = self._responseClass(transaction, strmOut) transaction.setResponse(response) return response of Application is giving the actual socket (strmOut) to the response class which I assume is HTTPResponse (though I can't find it). The constructor of HTTPResponse passes strmOut to the parent - Response - which sets the _strmOut instance variable. So it looks to me that trans.response()._strmOut should be the socket, yet when I attempt to verify that I get a TASASStramOut, and I have no clue where it came from. Perhaps someone more knowledgable can explain where I'm going wrong? -- / Peter Schuller, InfiDyne Technologies HB PGP userID: 0xE9758B7D or 'Peter Schuller <pet...@in...>' Key retrival: Send an E-Mail to get...@sc... E-Mail: pet...@in... Web: http://www.scode.org |
From: Ian B. <ia...@co...> - 2003-09-30 21:27:59
|
On Tuesday, September 30, 2003, at 04:23 PM, Peter Schuller wrote: >> for details) when the client closes the tcp connection prematurely, >> the >> python process/thread runniung the servlet starts looping with 100% >> cpu >> usage. > > Ok, I have confirmed that the problem occurrs with mod_webkit aswell. > > Also, I have been trying to understand what might be going on. But I > cannot > for the life of me find the location in the code where the data > actually gets > sent to the client. > > HTTPResponse.write() does write the data to _strmOut, but that is a > TASASStreamOut according to the debug output I put in before the > write(). The > TASASStreamOut class only puts data in an in-memory buffer (even when > flushing) and I cannot locate where that data is actually pulled out > of the > stream and sent to the socket. > > In either case, it is clear that > > def createResponseInTransaction(self, transaction, strmOut): > response = self._responseClass(transaction, strmOut) > transaction.setResponse(response) > return response > > of Application is giving the actual socket (strmOut) to the response > class > which I assume is HTTPResponse (though I can't find it). The > constructor of > HTTPResponse passes strmOut to the parent - Response - which sets the > _strmOut instance variable. So it looks to me that > trans.response()._strmOut > should be the socket, yet when I attempt to verify that I get a > TASASStramOut, and I have no clue where it came from. No, strmOut is TASASStreamOut -- that object is created by ThreadedAppServer, and Application never sees the raw socket object. TASASStreamOut does send data on the socket, though only if you do .autoCommit(1). The problem might be in TASASStreamOut.flush: try: sent = sent + self._socket.send(self._buffer[sent:sent+8192]) except socket.error, e: if e[0]==errno.EPIPE: #broken pipe pass else: print "StreamOut Error: ", e break This error handling is sub-optimal, to say the least. Ian |
From: Peter S. <pet...@in...> - 2003-09-30 21:56:48
|
> No, strmOut is TASASStreamOut -- that object is created by > ThreadedAppServer, and Application never sees the raw socket object. > TASASStreamOut does send data on the socket, though only if you do > .autoCommit(1). > > The problem might be in TASASStreamOut.flush: Ahhh! Thank you. For some reason I never looked at that method closely and went ahead to look at the base class. > try: > sent = sent + self._socket.send(self._buffer[sent:sent+8192]) > except socket.error, e: > if e[0]==errno.EPIPE: #broken pipe > pass > else: > print "StreamOut Error: ", e > break > > This error handling is sub-optimal, to say the least. Indeed. I just changed "pass" to "raise" - problem gone. Thanks again, -- / Peter Schuller, InfiDyne Technologies HB PGP userID: 0xE9758B7D or 'Peter Schuller <pet...@in...>' Key retrival: Send an E-Mail to get...@sc... E-Mail: pet...@in... Web: http://www.scode.org |