javtools-poolmngr — Architecture discussion regarding the db pool manager

Re: [Javtools-poolmngr] Requirements

[Dmitry T. <td...@qu...>]

Stefaan Delanghe wrote:

>5.A unused connection "sweeper", liken this to a garbase collector , unused
>connections or timed out connections not returned to the 
>   pool are regained.
>
I think that pool manager (PM) must has debug mode. In this mode PM have 
to keep a list of records when and who (this can be stack trace) get 
some rsource and didn't release it for some time period. This future 
will help to debug "resource leaks".

Re: [Javtools-poolmngr] Requirements added

[Dmitry T. <td...@qu...>]

Amit Deshpande wrote:

>1.Logging - Preliminary in the first version, advanced logging features
>could be looked into in a later version.
>   User has the option to enable/disable logging.
>
Propose use for logging LOG4J: 
http://jakarta.apache.org/log4j/docs/index.html

[Javtools-poolmngr] Chunking rethought

[Amit D. <ami...@ra...>]

Another use of Chunking ResultSets :
we could have internal chunking.
This comes to mind considering that we are gonna have a few physical jdbc
connections internally in the pool manager.
Instead of just replicating the user query which might return a huge
resultSet which will keep the Connection bust for a long time,we could
retrieve the data in chunks and then return the data completely to the user.
This will boost the overall performance of the PoolManager.
If the entire resultSet is to be returned to the user then internal caching
of the chunks of resultSets has to be done , which might fit into our object
model.
Issues regarding this have to be thought about though.
- Amit 

[Javtools-poolmngr] Requirement for Chunking

[Amit D. <ami...@ra...>]

Quoting Stefaan : 
"This could maybe proof a usefull utility in the db pool mngr.
In my opinion the use of resultset chunks is a bit obsolete. 
Since the most restrictions will be execute in a where clause of the
statement
which reflect the size of the resultset.
Why do you want to provide this utility and to what perpuse for the user?"

Steefan,
       Since we are going to have a thread based architecture.
How about this scenario :
 User specifies a chunk of 5 :
 We get the first chunk of 5 (using limit in the WHERE clause)
 return it to the user.
 (Assuming that the user now process this chunk)
 Meanwhile we have the next chunk ready in the thread.
 The user requests the next chunk and we give it to the user ,we already
have it ready.
 I am not to sure how the objects are gonna collaborate but we wud geta
clear picture during the technical design n object model.
Data retrieved is directly proportional to time.
So less data retrieved on each query and the same query being executed
repeatedly (which cud be a prepared query) Vs large data being retrieved in
a single query.
Also this might give rise to the issue of a timeout (If we have such a
criteria).The user might have 100000 records and request them in chunks of 5
, chunking doesnt seem good now.
Your opinion.
Regards,
Amit

[Javtools-poolmngr] Requirements added

[Amit D. <ami...@ra...>]

Hi All,
          Adding to the Requirements list : 
1.Logging - Preliminary in the first version, advanced logging features
could be looked into in a later version.
   User has the option to enable/disable logging.
2.ResultSets wrapped in collections like Vectors/ArrayLists.The reason as
pointed out by Stefaan the user doesnt have to import other 
   jdbc classes like ResultSets etc on his web page.
     I personally dont encourage vectors unless synchronization is
required.So we could have ArrayLists.
     Also we need to have methods as in jdbc's ResultSet like
getString(index), getInteger(index) otherwise the
   user will have to typecast values leading to a dynamic building code
,static binding is faster than a dynamic bind anyday.
   Thus to summarize we could have custom colelctions instead of ResultSets.
3.Provision for user to retrieve result sets in Chunks.We do not deliver the
entire resultSet , there needs to be a provision to deliever   
   the record in blocks of size 'n' ,'n' is stated by the user.Here
delievering the entire resultSet could be a default setting unless  
   "chunking" is set by the user.
  I need your opinion on this.
Plz comment.
- Amit

[Javtools-poolmngr] Requirements

[Stefaan D. <st...@ya...>]

Hi All ,

This mail is to initiate the design phase of the Pool Manager.
Before we move on the the architecture or the technical design , Stephen and
I thought we need to have the requirements listed first.
Stefaan you will need to edit this set of initial requirements this is a
very rough draft.
Others please feel free to contribute too.
Requirements
1.A PoolManager which could manage database pooling for 'n' databases.
2.Configuration of the Pool Manager needs to be in XML format.
3.Support for commit and rollback; user can handle this .Liken the situation
to setAutoCommit as false in jdbc, the user then has 
   control over the commit and rollback.
4.Support for isolation levels (this is database dependent but do give the
user to state an isolation level) if the underlying database   
   supports this isolation level.
5.A unused connection "sweeper", liken this to a garbase collector , unused
connections or timed out connections not returned to the 
   pool are regained.
6.Support for cached Prepared Statements.

   Also we could evaluate some existing pool managers (please do pass on the
links to such existing mangers) and list out the desired features.This could
lead us to a list of desired features missing in the existing pool managers
and help us build a better pool manager.
   I have used a pool Manager  www.protomatter.sourceforge.net .
   Need your opinion on the rqeuirements so that we can all agree and freeze
them.
   The next phase could be the technical architecture.

In addition to the above I would suggest

1. Support for multi database types.
2. A stack which holds the several transactions to be executed.
3. Thread with manipulates the stack.

Here are some usefull links in regard to pool managers:
http://www.looseleaf.net/looseleaf/lsihome.nsf/612ad3d77c83a706852567e7006b9a50/97e47c8b6e9ce59585256b19006415d1?OpenDocument

http://www.datadirect-technologies.com/download/docs/jdbc/jdbcref/pool.htm

http://www.inetsoftware.de/English/scripts/FrameSet.asp?path=http://www.inetsoftware.de/English/Produkte/OPTA/&name=ConnectionPooling.htm

http://www-106.ibm.com/developerworks/java/library/j-pool/

I would really appriciate that everybody posts his opinion in this forum regarding
his ideas for the sub-project Database Pool Manager

Regards,
Stefaan

=====
Stefaan Delanghe
ICQ: 110055343

__________________________________________________
Do You Yahoo!?
Yahoo! Health - Feel better, live better
http://health.yahoo.com

[Javtools-poolmngr] Requirements

[Amit D. <ami...@ra...>]

Hi All ,
         This mail is to initiate the design phase of the Pool Manager.
Before we move on the the architecture or the technical design , Stephen and
I thought we need to have the requirements listed first.
Stefaan you will need to edit this set of initial requirements this is a
very rough draft.
Others please feel free to contribute too.
Requirements
1.A PoolManager which could manage database pooling for 'n' databases.
2.Configuration of the Pool Manager needs to be in XML format.
3.Support for commit and rollback; user can handle this .Liken the situation
to setAutoCommit as false in jdbc, the user then has 
   control over the commit and rollback.
4.Support for isolation levels (this is database dependent but do give the
user to state an isolation level) if the underlying database   
   supports this isolation level.
5.A unused connection "sweeper", liken this to a garbase collector , unused
connections or timed out connections not returned to the 
   pool are regained.
6.Support for cached Prepared Statements.

   Also we could evaluate some existing pool managers (please do pass on the
links to such existing mangers) and list out the desired features.This could
lead us to a list of desired features missing in the existing pool managers
and help us build a better pool manager.
   I have used a pool Manager  www.protomatter.sourceforge.net .
   Need your opinion on the rqeuirements so that we can all agree and freeze
them.
   The next phase could be the technical architecture.
Regards,
Amit