proxool-developer

[Proxool-developer] Thread safety?

[Steve B. <st...@te...>]

Hello,

 

I'm using the Proxool DataSource support within another open source project,
QuickFIX/J. Some of my users are reporting a problem that results in
exceptions like the following.

 

java.sql.SQLException: We are already in the process of making 1 connections
and the number of simultaneous 

builds has been throttled to 0

 

I am explicitly setting the simultaneous build throttle value to 10
(although that's supposed to be the default) and the error still occurs at
times. I'm wondering if it's possible that the configuration information is
not stored in a thread-safe manner (that would explain the "stale" zero
value). I see that the configuration properties are not final (makes sense)
and also not synchronized or declared volatile. Are the throttling
configurations potentially accessed by multiple threads? If so, shouldn't
they be synchronized or volatile?

 

This error has been reported with older versions of Proxool. I've upgraded
to the latest version, but I noticed the configurations are still not
synchronized or volatile so I'm wondering if this might still be an issue.

 

Thanks for any help you can provide.

 

Steve

 

Re: [Proxool-developer] Thread safety?

[Bill H. <bi...@lo...>]

Hi Steve,

We do pay a lot of attention to thread safety and have made us of Doug Lea's
concurrency libraries to solve that problem. It's possible that there is a
bug, sure, but it's something we've thought a lot about and haven't had
problems with before. (I mean, specifically the simultaneous build
throttle).

Can you tell me how you are configuring Proxool and how you are using it?
Also, your log file should output the configuration (if you make it
appropriately verbose). It would be interesting to see whether you have more
pools than you expect - or if that error occurs before the pool has been
configured.

-
Bill

Re: [Proxool-developer] Thread safety?

[Steve B. <st...@te...>]

> We do pay a lot of attention to thread safety and have made us of Doug
> Lea's
> concurrency libraries to solve that problem. It's possible that there
> is a bug, sure, but it's something we've thought a lot about and
> haven't had problems with before. (I mean, specifically the
> simultaneous build throttle).
>
> Can you tell me how you are configuring Proxool and how you are using it?

Hi Bill,

First, my users have seen this problem but I haven't personally seen it.
I want to do some more investigation about the version of QuickFIX/J they
are using. A user posted a bug related to the simultaneous build throttle
setting in 0.9.0RC2 and we worked around it. However, the user may be using
a version prior to the fix.

http://www.quickfixj.org/jira/browse/QFJ-316

As for how we use Proxool, it's pretty basic. We create a data source, set
some bean properties to configure it, and then proceed to use it.

> Also, your log file should output the configuration (if you make it
> appropriately verbose). It would be interesting to see whether you have
> more pools than you expect - or if that error occurs before the pool
> has been configured.

I'll need to ask my user to do this.

Looking at the 0.9.1, I'm not sure how the configuration could be thread
safe. The settings are stored in unprotected member fields. The data
source
getConnection() method can be called from multiple threads and those
configuration fields might not be consistent across processor caches.

I'm also wondering about the comment in the FastArrayList used to store
the pooled connections. It says the implementation is not cross-platform
and may fail unexpectedly on some architectures (presumedly due to
insufficient synchronization). If "some architectures" means multicore or
multiprocessor architectures, those are becoming very common. I noticed
the comment is from version 0.6. Is this not a concern?

Regards,

Steve

Re: [Proxool-developer] Thread safety?

[Steve B. <st...@te...>]

> We do pay a lot of attention to thread safety and have made us of Doug
> Lea's
> concurrency libraries to solve that problem. It's possible that there
> is a bug, sure, but it's something we've thought a lot about and
> haven't had problems with before. (I mean, specifically the
> simultaneous build throttle).
>
> Can you tell me how you are configuring Proxool and how you are using it?

Hi Bill,

First, my users have seen this problem but I haven't personally seen it.
I want to do some more investigation about the version of QuickFIX/J they
are using. A user posted a bug related to the simultaneous build throttle
setting in 0.9.0RC2 and we worked around it. However, the user may be using
a version prior to the fix.

http://www.quickfixj.org/jira/browse/QFJ-316

As for how we use Proxool, it's pretty basic. We create a data source, set
some bean properties to configure it, and then proceed to use it.

> Also, your log file should output the configuration (if you make it
> appropriately verbose). It would be interesting to see whether you have
> more pools than you expect - or if that error occurs before the pool
> has been configured.

I'll need to ask my user to do this.

Looking at the 0.9.1, I'm not sure how the configuration could be thread
safe. The settings are stored in unprotected member fields. The data
source
getConnection() method can be called from multiple threads and those
configuration fields might not be consistent across processor caches.

I'm also wondering about the comment in the FastArrayList used to store
the pooled connections. It says the implementation is not cross-platform
and may fail unexpectedly on some architectures (presumedly due to
insufficient synchronization). If "some architectures" means multicore or
multiprocessor architectures, those are becoming very common. I noticed
the comment is from version 0.6. Is this not a concern?

Regards,

Steve

Re: [Proxool-developer] Thread safety?

[Bill H. <bi...@lo...>]

Hi Steve,

Sorry for the slow response. I think I'd like to see a test case running
with QuickFIX/J that reproduces this problem. It's pretty tricky to solve
this indirectly - there are just too many variables. And the log output
would be so useful.

-
Bill