Webware for Python

Patches item #1806696, was opened at 2007-10-03 05:25
Message generated for change (Comment added) made by cito
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: WebKit
Group: None
>Status: Open
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Steve Schwarz (saschwarz)
Assigned to: Christoph Zwerschke (cito)
Summary: Transaction Monitoring and Control

Initial Comment:
Hi,
I've been running a local patch for a couple weeks now with following features:

1 - Added two methods to ThreadedAppServer: startRequest() and endRequest() methods. These methods can be used to implement page statistics and/or click tracking by session. Default implementation supports feature 2 below.

2 - Allow real time monitoring of the requests being processed by each thread. A new ThreadControl page has been added to the Admin context. Setting 'TrackThreads' to True in AppServer.config enables this feature. If the supplied thread2.py module is loaded requests can be cancelled via this page.

3 - Support automatically cancelling requests taking more than a configurable amount of time. Setting 'CancelLongRequests' to the number of seconds after which a request should be cancelled in the AppServer.config file enables this feature. This feature also requires 'TrackThreads' to be enabled.

The patch is relatively small, if there is interest in looking at these changes and moving them into the mainline I'd be happy to forward them to anyone interested. Of course, comments and suggestions are very welcome.

The performance penalty for feature 1 is the cost of some accessors into the current transaction, a couple time() calls and two function calls. I would guess these changes should have a negligible impact on performance.

Features 2 and 3 use a new class which is basically a locked dictionary. So the cost is two additional mutex lock/unlocks for each transaction. If WebKit didn't support adding/removing threads dynamically that mutex could be removed and the additional overhead is some dictionary accesses.

I'd be happy to get feedback on these enhancements.

Thanks,
Steve

----------------------------------------------------------------------

>Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 21:33

Message:
Logged In: YES 
user_id=193957
Originator: NO

I just noticed that I have only implemented features 1 and 2, and
completely forgot about your CancelLongRequests option for canceling
long-runners automatically. Reopened this ticket as a reminder to implement
that, too.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 20:47

Message:
Logged In: YES 
user_id=193957
Originator: NO

Thanks a lot, Steve. This has now been implemented in r7097 and will go
into Webware 0.9.5. I only changed some implementation details. For
instance, I am not using any locks. Since we have only one writer, and
since setattr operations are atomic in Python, you can actually do without
them. I have also added an improved version of your ThreadControl servlet.
Let me know if you think this implementation is ok or if you find any bugs
or problems.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Patches item #1806696, was opened at 2007-10-03 05:25
Message generated for change (Comment added) made by cito
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: WebKit
Group: None
>Status: Closed
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Steve Schwarz (saschwarz)
Assigned to: Christoph Zwerschke (cito)
Summary: Transaction Monitoring and Control

Initial Comment:
Hi,
I've been running a local patch for a couple weeks now with following features:

1 - Added two methods to ThreadedAppServer: startRequest() and endRequest() methods. These methods can be used to implement page statistics and/or click tracking by session. Default implementation supports feature 2 below.

2 - Allow real time monitoring of the requests being processed by each thread. A new ThreadControl page has been added to the Admin context. Setting 'TrackThreads' to True in AppServer.config enables this feature. If the supplied thread2.py module is loaded requests can be cancelled via this page.

3 - Support automatically cancelling requests taking more than a configurable amount of time. Setting 'CancelLongRequests' to the number of seconds after which a request should be cancelled in the AppServer.config file enables this feature. This feature also requires 'TrackThreads' to be enabled.

The patch is relatively small, if there is interest in looking at these changes and moving them into the mainline I'd be happy to forward them to anyone interested. Of course, comments and suggestions are very welcome.

The performance penalty for feature 1 is the cost of some accessors into the current transaction, a couple time() calls and two function calls. I would guess these changes should have a negligible impact on performance.

Features 2 and 3 use a new class which is basically a locked dictionary. So the cost is two additional mutex lock/unlocks for each transaction. If WebKit didn't support adding/removing threads dynamically that mutex could be removed and the additional overhead is some dictionary accesses.

I'd be happy to get feedback on these enhancements.

Thanks,
Steve

----------------------------------------------------------------------

>Comment By: Christoph Zwerschke (cito)
Date: 2007-11-25 15:58

Message:
Logged In: YES 
user_id=193957
Originator: NO

In r7107 has now been implemented in r7107. Again, let me know if there is
still something missing or not working for you.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 21:33

Message:
Logged In: YES 
user_id=193957
Originator: NO

I just noticed that I have only implemented features 1 and 2, and
completely forgot about your CancelLongRequests option for canceling
long-runners automatically. Reopened this ticket as a reminder to implement
that, too.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 20:47

Message:
Logged In: YES 
user_id=193957
Originator: NO

Thanks a lot, Steve. This has now been implemented in r7097 and will go
into Webware 0.9.5. I only changed some implementation details. For
instance, I am not using any locks. Since we have only one writer, and
since setattr operations are atomic in Python, you can actually do without
them. I have also added an improved version of your ThreadControl servlet.
Let me know if you think this implementation is ok or if you find any bugs
or problems.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Patches item #1806696, was opened at 2007-10-02 22:25
Message generated for change (Comment added) made by saschwarz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: WebKit
Group: None
Status: Closed
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Steve Schwarz (saschwarz)
Assigned to: Christoph Zwerschke (cito)
Summary: Transaction Monitoring and Control

Initial Comment:
Hi,
I've been running a local patch for a couple weeks now with following features:

1 - Added two methods to ThreadedAppServer: startRequest() and endRequest() methods. These methods can be used to implement page statistics and/or click tracking by session. Default implementation supports feature 2 below.

2 - Allow real time monitoring of the requests being processed by each thread. A new ThreadControl page has been added to the Admin context. Setting 'TrackThreads' to True in AppServer.config enables this feature. If the supplied thread2.py module is loaded requests can be cancelled via this page.

3 - Support automatically cancelling requests taking more than a configurable amount of time. Setting 'CancelLongRequests' to the number of seconds after which a request should be cancelled in the AppServer.config file enables this feature. This feature also requires 'TrackThreads' to be enabled.

The patch is relatively small, if there is interest in looking at these changes and moving them into the mainline I'd be happy to forward them to anyone interested. Of course, comments and suggestions are very welcome.

The performance penalty for feature 1 is the cost of some accessors into the current transaction, a couple time() calls and two function calls. I would guess these changes should have a negligible impact on performance.

Features 2 and 3 use a new class which is basically a locked dictionary. So the cost is two additional mutex lock/unlocks for each transaction. If WebKit didn't support adding/removing threads dynamically that mutex could be removed and the additional overhead is some dictionary accesses.

I'd be happy to get feedback on these enhancements.

Thanks,
Steve

----------------------------------------------------------------------

>Comment By: Steve Schwarz (saschwarz)
Date: 2007-11-25 12:32

Message:
Logged In: YES 
user_id=1111832
Originator: YES

Thanks for promoting this code. I'll take a look at it this week.

My concern with removing the mutexes is the situation where the list of
active requests is being created for a request (i.e. for the admin screen)
at the same time the number of threads is being reduced. I believe I had
the map iterator raise an exception when the map changed underneath it in
that situation.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-25 08:58

Message:
Logged In: YES 
user_id=193957
Originator: NO

In r7107 has now been implemented in r7107. Again, let me know if there is
still something missing or not working for you.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 14:33

Message:
Logged In: YES 
user_id=193957
Originator: NO

I just noticed that I have only implemented features 1 and 2, and
completely forgot about your CancelLongRequests option for canceling
long-runners automatically. Reopened this ticket as a reminder to implement
that, too.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 13:47

Message:
Logged In: YES 
user_id=193957
Originator: NO

Thanks a lot, Steve. This has now been implemented in r7097 and will go
into Webware 0.9.5. I only changed some implementation details. For
instance, I am not using any locks. Since we have only one writer, and
since setattr operations are atomic in Python, you can actually do without
them. I have also added an improved version of your ThreadControl servlet.
Let me know if you think this implementation is ok or if you find any bugs
or problems.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Patches item #1806696, was opened at 2007-10-03 05:25
Message generated for change (Comment added) made by cito
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: WebKit
Group: None
Status: Closed
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Steve Schwarz (saschwarz)
Assigned to: Christoph Zwerschke (cito)
Summary: Transaction Monitoring and Control

Initial Comment:
Hi,
I've been running a local patch for a couple weeks now with following features:

1 - Added two methods to ThreadedAppServer: startRequest() and endRequest() methods. These methods can be used to implement page statistics and/or click tracking by session. Default implementation supports feature 2 below.

2 - Allow real time monitoring of the requests being processed by each thread. A new ThreadControl page has been added to the Admin context. Setting 'TrackThreads' to True in AppServer.config enables this feature. If the supplied thread2.py module is loaded requests can be cancelled via this page.

3 - Support automatically cancelling requests taking more than a configurable amount of time. Setting 'CancelLongRequests' to the number of seconds after which a request should be cancelled in the AppServer.config file enables this feature. This feature also requires 'TrackThreads' to be enabled.

The patch is relatively small, if there is interest in looking at these changes and moving them into the mainline I'd be happy to forward them to anyone interested. Of course, comments and suggestions are very welcome.

The performance penalty for feature 1 is the cost of some accessors into the current transaction, a couple time() calls and two function calls. I would guess these changes should have a negligible impact on performance.

Features 2 and 3 use a new class which is basically a locked dictionary. So the cost is two additional mutex lock/unlocks for each transaction. If WebKit didn't support adding/removing threads dynamically that mutex could be removed and the additional overhead is some dictionary accesses.

I'd be happy to get feedback on these enhancements.

Thanks,
Steve

----------------------------------------------------------------------

>Comment By: Christoph Zwerschke (cito)
Date: 2007-11-25 21:24

Message:
Logged In: YES 
user_id=193957
Originator: NO

I don't think that the changing map iteration is a problem since I'm using
the items() or values() of the map only.

However, now that you say it, I see a different problem. There is a
minuscule chance that while the list of active threads is iterated, one of
these threads finishes by itself and processes another request. In this
case, the new request will be canceled instead of the one we actually
wanted to cancel. I think both of our implementations have this problem. To
solve this, we may need a different kind of lock used inside the
threadloop(). I'll examine this end of this week. Maybe you can make some
suggestions until then already.

----------------------------------------------------------------------

Comment By: Steve Schwarz (saschwarz)
Date: 2007-11-25 19:32

Message:
Logged In: YES 
user_id=1111832
Originator: YES

Thanks for promoting this code. I'll take a look at it this week.

My concern with removing the mutexes is the situation where the list of
active requests is being created for a request (i.e. for the admin screen)
at the same time the number of threads is being reduced. I believe I had
the map iterator raise an exception when the map changed underneath it in
that situation.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-25 15:58

Message:
Logged In: YES 
user_id=193957
Originator: NO

In r7107 has now been implemented in r7107. Again, let me know if there is
still something missing or not working for you.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 21:33

Message:
Logged In: YES 
user_id=193957
Originator: NO

I just noticed that I have only implemented features 1 and 2, and
completely forgot about your CancelLongRequests option for canceling
long-runners automatically. Reopened this ticket as a reminder to implement
that, too.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 20:47

Message:
Logged In: YES 
user_id=193957
Originator: NO

Thanks a lot, Steve. This has now been implemented in r7097 and will go
into Webware 0.9.5. I only changed some implementation details. For
instance, I am not using any locks. Since we have only one writer, and
since setattr operations are atomic in Python, you can actually do without
them. I have also added an improved version of your ThreadControl servlet.
Let me know if you think this implementation is ok or if you find any bugs
or problems.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Patches item #1806696, was opened at 2007-10-02 22:25
Message generated for change (Comment added) made by saschwarz
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: WebKit
Group: None
Status: Closed
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Steve Schwarz (saschwarz)
Assigned to: Christoph Zwerschke (cito)
Summary: Transaction Monitoring and Control

Initial Comment:
Hi,
I've been running a local patch for a couple weeks now with following features:

1 - Added two methods to ThreadedAppServer: startRequest() and endRequest() methods. These methods can be used to implement page statistics and/or click tracking by session. Default implementation supports feature 2 below.

2 - Allow real time monitoring of the requests being processed by each thread. A new ThreadControl page has been added to the Admin context. Setting 'TrackThreads' to True in AppServer.config enables this feature. If the supplied thread2.py module is loaded requests can be cancelled via this page.

3 - Support automatically cancelling requests taking more than a configurable amount of time. Setting 'CancelLongRequests' to the number of seconds after which a request should be cancelled in the AppServer.config file enables this feature. This feature also requires 'TrackThreads' to be enabled.

The patch is relatively small, if there is interest in looking at these changes and moving them into the mainline I'd be happy to forward them to anyone interested. Of course, comments and suggestions are very welcome.

The performance penalty for feature 1 is the cost of some accessors into the current transaction, a couple time() calls and two function calls. I would guess these changes should have a negligible impact on performance.

Features 2 and 3 use a new class which is basically a locked dictionary. So the cost is two additional mutex lock/unlocks for each transaction. If WebKit didn't support adding/removing threads dynamically that mutex could be removed and the additional overhead is some dictionary accesses.

I'd be happy to get feedback on these enhancements.

Thanks,
Steve

----------------------------------------------------------------------

>Comment By: Steve Schwarz (saschwarz)
Date: 2007-11-26 21:30

Message:
Logged In: YES 
user_id=1111832
Originator: YES

Christoph: "However, now that you say it, I see a different problem. There
is a
minuscule chance that while the list of active threads is iterated, one
of
these threads finishes by itself and processes another request. In this
case, the new request will be canceled instead of the one we actually
wanted to cancel. I think both of our implementations have this problem.
To
solve this, we may need a different kind of lock used inside the
threadloop()."

I only just started looking at the code. One thing that comes to mind is
to have the exception that is passed into ThreadWorker.abort() contain the
requestId being cancelled. Then that code could compare request ids just
before issuing the async thread cancel.

Another idea I played with was having long running tasks and/or cancelled
tasks give a traceback when they are cancelled. That way you can see what
the page was working on when it was killed - the client doesn't have to get
the traceback though. In my original version it did that, the version I
supplied in this ticket catches that specific exception and just logged a
message. WDYT?

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-25 14:24

Message:
Logged In: YES 
user_id=193957
Originator: NO

I don't think that the changing map iteration is a problem since I'm using
the items() or values() of the map only.

However, now that you say it, I see a different problem. There is a
minuscule chance that while the list of active threads is iterated, one of
these threads finishes by itself and processes another request. In this
case, the new request will be canceled instead of the one we actually
wanted to cancel. I think both of our implementations have this problem. To
solve this, we may need a different kind of lock used inside the
threadloop(). I'll examine this end of this week. Maybe you can make some
suggestions until then already.

----------------------------------------------------------------------

Comment By: Steve Schwarz (saschwarz)
Date: 2007-11-25 12:32

Message:
Logged In: YES 
user_id=1111832
Originator: YES

Thanks for promoting this code. I'll take a look at it this week.

My concern with removing the mutexes is the situation where the list of
active requests is being created for a request (i.e. for the admin screen)
at the same time the number of threads is being reduced. I believe I had
the map iterator raise an exception when the map changed underneath it in
that situation.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-25 08:58

Message:
Logged In: YES 
user_id=193957
Originator: NO

In r7107 has now been implemented in r7107. Again, let me know if there is
still something missing or not working for you.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 14:33

Message:
Logged In: YES 
user_id=193957
Originator: NO

I just noticed that I have only implemented features 1 and 2, and
completely forgot about your CancelLongRequests option for canceling
long-runners automatically. Reopened this ticket as a reminder to implement
that, too.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 13:47

Message:
Logged In: YES 
user_id=193957
Originator: NO

Thanks a lot, Steve. This has now been implemented in r7097 and will go
into Webware 0.9.5. I only changed some implementation details. For
instance, I am not using any locks. Since we have only one writer, and
since setattr operations are atomic in Python, you can actually do without
them. I have also added an improved version of your ThreadControl servlet.
Let me know if you think this implementation is ok or if you find any bugs
or problems.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Patches item #1806696, was opened at 2007-10-03 05:25
Message generated for change (Comment added) made by cito
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: WebKit
Group: None
Status: Closed
Resolution: Accepted
Priority: 5
Private: No
Submitted By: Steve Schwarz (saschwarz)
Assigned to: Christoph Zwerschke (cito)
Summary: Transaction Monitoring and Control

Initial Comment:
Hi,
I've been running a local patch for a couple weeks now with following features:

1 - Added two methods to ThreadedAppServer: startRequest() and endRequest() methods. These methods can be used to implement page statistics and/or click tracking by session. Default implementation supports feature 2 below.

2 - Allow real time monitoring of the requests being processed by each thread. A new ThreadControl page has been added to the Admin context. Setting 'TrackThreads' to True in AppServer.config enables this feature. If the supplied thread2.py module is loaded requests can be cancelled via this page.

3 - Support automatically cancelling requests taking more than a configurable amount of time. Setting 'CancelLongRequests' to the number of seconds after which a request should be cancelled in the AppServer.config file enables this feature. This feature also requires 'TrackThreads' to be enabled.

The patch is relatively small, if there is interest in looking at these changes and moving them into the mainline I'd be happy to forward them to anyone interested. Of course, comments and suggestions are very welcome.

The performance penalty for feature 1 is the cost of some accessors into the current transaction, a couple time() calls and two function calls. I would guess these changes should have a negligible impact on performance.

Features 2 and 3 use a new class which is basically a locked dictionary. So the cost is two additional mutex lock/unlocks for each transaction. If WebKit didn't support adding/removing threads dynamically that mutex could be removed and the additional overhead is some dictionary accesses.

I'd be happy to get feedback on these enhancements.

Thanks,
Steve

----------------------------------------------------------------------

>Comment By: Christoph Zwerschke (cito)
Date: 2007-11-29 12:03

Message:
Logged In: YES 
user_id=193957
Originator: NO

Concerning the first problem, checking the thread again just before it's
canceled would decrease the (already small) possibility of a wrong
cancelation, but the chance would still exist. I've now checked in an
air-tight solution in r7112 which should not cost performance and makes
sure that always the right request is canceled. 

Concerning your second idea: Currently requests are aborted with an
ConnectionAbortedError. Actually this exception is used in the case where
the client aborted the connection and the response could not be sent to the
client any more.

I think you're right and a different exception should be used for aborting
requests, e.g. a new class RequestAbortedError. This should be handled just
like any Exception, i.e. a traceback is logged and the client gets either
the traceback or an error message (depending on the ShowDebugInfoOnErrors
setting) if the servlet does not catch this. If you agree, I will implement
it that way and we can close this issue.

Also, what do you think about the default setting of MaxRequestTime = 300.
Is this ok? it corresponds with the Apache default timeout value. Or should
we set it to None (disable the feature) by default? 

----------------------------------------------------------------------

Comment By: Steve Schwarz (saschwarz)
Date: 2007-11-27 04:30

Message:
Logged In: YES 
user_id=1111832
Originator: YES

Christoph: "However, now that you say it, I see a different problem. There
is a
minuscule chance that while the list of active threads is iterated, one
of
these threads finishes by itself and processes another request. In this
case, the new request will be canceled instead of the one we actually
wanted to cancel. I think both of our implementations have this problem.
To
solve this, we may need a different kind of lock used inside the
threadloop()."

I only just started looking at the code. One thing that comes to mind is
to have the exception that is passed into ThreadWorker.abort() contain the
requestId being cancelled. Then that code could compare request ids just
before issuing the async thread cancel.

Another idea I played with was having long running tasks and/or cancelled
tasks give a traceback when they are cancelled. That way you can see what
the page was working on when it was killed - the client doesn't have to get
the traceback though. In my original version it did that, the version I
supplied in this ticket catches that specific exception and just logged a
message. WDYT?

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-25 21:24

Message:
Logged In: YES 
user_id=193957
Originator: NO

I don't think that the changing map iteration is a problem since I'm using
the items() or values() of the map only.

However, now that you say it, I see a different problem. There is a
minuscule chance that while the list of active threads is iterated, one of
these threads finishes by itself and processes another request. In this
case, the new request will be canceled instead of the one we actually
wanted to cancel. I think both of our implementations have this problem. To
solve this, we may need a different kind of lock used inside the
threadloop(). I'll examine this end of this week. Maybe you can make some
suggestions until then already.

----------------------------------------------------------------------

Comment By: Steve Schwarz (saschwarz)
Date: 2007-11-25 19:32

Message:
Logged In: YES 
user_id=1111832
Originator: YES

Thanks for promoting this code. I'll take a look at it this week.

My concern with removing the mutexes is the situation where the list of
active requests is being created for a request (i.e. for the admin screen)
at the same time the number of threads is being reduced. I believe I had
the map iterator raise an exception when the map changed underneath it in
that situation.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-25 15:58

Message:
Logged In: YES 
user_id=193957
Originator: NO

In r7107 has now been implemented in r7107. Again, let me know if there is
still something missing or not working for you.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 21:33

Message:
Logged In: YES 
user_id=193957
Originator: NO

I just noticed that I have only implemented features 1 and 2, and
completely forgot about your CancelLongRequests option for canceling
long-runners automatically. Reopened this ticket as a reminder to implement
that, too.

----------------------------------------------------------------------

Comment By: Christoph Zwerschke (cito)
Date: 2007-11-23 20:47

Message:
Logged In: YES 
user_id=193957
Originator: NO

Thanks a lot, Steve. This has now been implemented in r7097 and will go
into Webware 0.9.5. I only changed some implementation details. For
instance, I am not using any locks. Since we have only one writer, and
since setattr operations are atomic in Python, you can actually do without
them. I have also added an improved version of your ThreadControl servlet.
Let me know if you think this implementation is ok or if you find any bugs
or problems.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=304866&aid=1806696&group_id=4866

Hi Gregory,

> Here's the code I'm using, which I believe it set up correctly
> following the docs, but I'm hitting the following errors:
> 
> Error 1:
> 
> Exception exceptions.TypeError: "'NoneType' object is not callable" in
> <bound method PooledDB.__del__ of <DBUtils.PooledDB.PooledDB instance
> at 0x85095cc>> ignored

In which situation do you get this error? I could not reproduce this.

> Error 2:
> 
> ...
> [Tue Nov 20 17:07:23 2007] [error] [client 127.0.0.1]
> OperationalError: (1040, 'Too many connections')
> 
> I set my.conf to have 16 max connections so I don't know why it says
> there are too many since I only gave the pool 10.

There are two reasons why this can happen.

First, make sure you are setting up POOL only one time, globally. Put a 
print statement after the "POOL = " statement and make sure it prints 
only once.

Second, in your POOL object, you don't allow shared connections 
(maxshared = 0), and in your query function you are also requesting 
dedicated connections. Now if you are calling the function very 
frequently from many threads (and the query time is longer than this 
frequency), then the following will happen: The POOL will first give 
away the 4 connections you put in your pool initially. Then there are no 
connections in the pool any more, and DBUtils will request new 
connections from the database until you reach maxconnections=10, at 
which point it will raise an error, because you have set blocking=False.

As a solution, you can set blocking=True, i.e. DBUtils waits until 
another thread has released its connection so it can be reused. This 
will never throw an exception, but will of course not give the best 
performance it you are not allowing for enough connections.

To improve performance, you can increase the max number of connections 
of MySQL to 100 or so, and set maxconnections a little bit lower (so you 
can still log in as an administrator).

A better solution would be to use shared connections (setting maxshared 
to a positive value), but this works only with a database module that 
provides thread-safe connections. MySQLdb will probably not do it (check 
MySQLdb.threadsafety to be sure, it should be > 1 to allow this, if it's 
not, then DBUtils will assume maxshared = 0 no matter how you set it).

You can also try PersistentDB instead of PooledDB and check whether it 
gives better performance for you.

-- Chris

On Nov 20, 2007 8:11 PM, Christoph Zwerschke <cito@...> wrote:
> Hi Gregory,
>
> > Here's the code I'm using, which I believe it set up correctly
> > following the docs, but I'm hitting the following errors:
> >
> > Error 1:
> >
> > Exception exceptions.TypeError: "'NoneType' object is not callable" in
> > <bound method PooledDB.__del__ of <DBUtils.PooledDB.PooledDB instance
> > at 0x85095cc>> ignored
>
> In which situation do you get this error? I could not reproduce this.

It just started appearing in my apache error log file.  I can't really
tell where it's coming from.  I guess I'll worry about that one later.

I also see that message for another unrelated module (flup).  Perhaps
it's something mod_wsgi is doing.

>
> > Error 2:
> >
> > ...
> > [Tue Nov 20 17:07:23 2007] [error] [client 127.0.0.1]
> > OperationalError: (1040, 'Too many connections')
> >
> > I set my.conf to have 16 max connections so I don't know why it says
> > there are too many since I only gave the pool 10.
>
> There are two reasons why this can happen.
>
> First, make sure you are setting up POOL only one time, globally. Put a
> print statement after the "POOL = " statement and make sure it prints
> only once.

So I'm using Apache (MPM=Prefork) with mod_wsgi and web.py.  So after
further research it looks like, each Apache child process is going to
have it's own global variables.  So perhaps it's not really feasible
for me to have a pool in this setup.

I found child process stuff from reading here:
http://code.google.com/p/modwsgi/wiki/ProcessesAndThreading

> Second, in your POOL object, you don't allow shared connections
> (maxshared = 0), and in your query function you are also requesting
> dedicated connections. Now if you are calling the function very
> frequently from many threads (and the query time is longer than this
> frequency), then the following will happen: The POOL will first give
> away the 4 connections you put in your pool initially. Then there are no
> connections in the pool any more, and DBUtils will request new
> connections from the database until you reach maxconnections=10, at
> which point it will raise an error, because you have set blocking=False.

My error seems to be coming from MySQL though, and not from DBUtils.
It's probably my last point that each apache process is actually
making its own pool.

> As a solution, you can set blocking=True, i.e. DBUtils waits until
> another thread has released its connection so it can be reused. This
> will never throw an exception, but will of course not give the best
> performance it you are not allowing for enough connections.
>
> To improve performance, you can increase the max number of connections
> of MySQL to 100 or so, and set maxconnections a little bit lower (so you
> can still log in as an administrator).

I'm on a pretty low memory machine (300MB) but I could try going a bit
higher on the connections.

> A better solution would be to use shared connections (setting maxshared
> to a positive value), but this works only with a database module that
> provides thread-safe connections. MySQLdb will probably not do it (check
> MySQLdb.threadsafety to be sure, it should be > 1 to allow this, if it's
> not, then DBUtils will assume maxshared = 0 no matter how you set it).

I got:

>>> MySQLdb.threadsafety
1

So that means it's not thread safe?  So no sharing?


> You can also try PersistentDB instead of PooledDB and check whether it
> gives better performance for you.

So I'm thinking either to go with PersistentDB and then have max mysql
connections = max apache servers - 1.

Or I could put a small pool in each child.  So say a pool of 3, and
then have max mysql connections = max apache servers * 3

Any ideas which would give better performance offhand?

Thanks again for all the help.  I think I'm starting to figure this stuff out.

-Greg

Gregory Piñero wrote:
>>> [Tue Nov 20 17:07:23 2007] [error] [client 127.0.0.1]
>>> OperationalError: (1040, 'Too many connections')
>>>
>>> I set my.conf to have 16 max connections so I don't know why it says
>>> there are too many since I only gave the pool 10.
>> There are two reasons why this can happen.
>>
>> First, make sure you are setting up POOL only one time, globally. Put a
>> print statement after the "POOL = " statement and make sure it prints
>> only once.
> 
> So I'm using Apache (MPM=Prefork) with mod_wsgi and web.py.  So after
> further research it looks like, each Apache child process is going to
> have it's own global variables.  So perhaps it's not really feasible
> for me to have a pool in this setup.

Yes, mod_python and mod_wsgi run in the child processes that Apache is 
spawning with the Prefork module. So you are actually living in a 
multi-processing environment here, not a multi-threading environment.

DBUtils has originally been written for multi-threading environments 
such as Webware's ThreadedAppServer (which is not directly coupled to 
Apache's processes even if you use the Apache mod_webkit module). So I 
haven't really thought about multi-processing environments yet.

But that does not mean that DBUtils is not suited here. Each process 
will then have its own pool that you must make accordingly smaller (the 
maximum number of pools will be MaxClients/ThreadsPerChild). Only if 
ThreadsPerChild is very low or even=1 (as I think it is in the Prefork 
module), then using DBUtils will be of no use.

I'll probably need to add something about multi-processing in the 
DBUtils documentation, as we are going more into the direction of 
multi-processing with Python, due to the new multi-core CPUs and the old 
Python GIL problem.

> My error seems to be coming from MySQL though, and not from DBUtils.
> It's probably my last point that each apache process is actually
> making its own pool.

Yes, you're right, this error message is coming from MySQL. I should 
have looked more carefully, since DBUtils would have raised a different 
exception. Each of your Apache processes creates its own pool of 
initially 4 connections, and the 5th process will get into troubles 
then. Plus, you cannot make use of the pools at all, since the processes 
are not creating extra threads.

> I got:
> 
>>>> MySQLdb.threadsafety
> 1
> 
> So that means it's not thread safe?  So no sharing?

Yes. No sharing possible then.

>> You can also try PersistentDB instead of PooledDB and check whether it
>> gives better performance for you.
> 
> So I'm thinking either to go with PersistentDB and then have max mysql
> connections = max apache servers - 1.

PersistentDB will not be of great use either as long as you are using 
the Prefork MPM, since there you will have only one Thread per child.

One way to go would be to switch over to a different MPM and use small 
PooledDB pools or PersistentDB.

I'll think about other solutions in the multi-process environment. 
Probably we can also find something in the Internet, I haven't googled 
yet. If you find something, let me know so I can mention it in the 
DBUtils documentation.

-- Chris

On Nov 21, 2007 3:09 AM, Christoph Zwerschke <cito@...> wrote:
> PersistentDB will not be of great use either as long as you are using
> the Prefork MPM, since there you will have only one Thread per child.
>
> One way to go would be to switch over to a different MPM and use small
> PooledDB pools or PersistentDB.
>
> I'll think about other solutions in the multi-process environment.
> Probably we can also find something in the Internet, I haven't googled
> yet. If you find something, let me know so I can mention it in the
> DBUtils documentation.
>

Thanks for the help.  Let me know what you find when you get a chance
to look around.  I went ahead and put in PersistentDB for now.  I
doesn't seem to hurt performance.

-Greg