Re: [Classifier4j-devel] Update Word Probability Break Down

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Is it correct to say that our database connection is getting setup and torn 
down each time updateWordProbability is called?

From what I gather, this is not good practice to begin with.  Opening and 
closing a database connection 60-80 times per second has to be taxing.  As I 
understand it, this is where connection pooling comes in.

I wonder if JDBC might have some protection mechanism build in for clients 
that go haywire.  Perhaps it closes connections for processes that open and 
close connections too many times.  Maybe it just fails.

AH HA!  This is a diffence between my dbTest.java and connect.java.  I am not 
connecting and disconnecting on each record.  I will rebuild this to test.

I don't know the first thing about how to implement connection pooling to 
begin with, much less in this conext, but I guess that's what I'll start 
working on!

BTW, I've narrowed the error to the call to connectionManager.getConnection() 
in updateWordProbability.  I have increased the exception handling to produce 
the following information:

SQLState: 08S01
VendorError: 0
NextException: null

SQLState 08S01 = mySQL error ER_BAD_HOST_ERROR according to:

http://mysql.mirror.trueserver.nl/doc/en/Error-returns.html

Matt Collier
RemoteIT
mco...@my...
877-4-NEW-LAN

-----Original Message-----
From: "Matt Collier" <MCo...@my...>
To: cla...@li...
Date: Wed, 12 Nov 2003 09:34:08 -0600
Subject: [Classifier4j-devel] Update Word Probability Break Down

> Hi Nick, yes I am using the latest CVS code.
> 
> How did you determine that the problem resides in the createTable function?
> 
> Have you been able to reproduce the problem?
> 
> I am not catching an exception there, I'm catching it in the 
> updateWordProbability. 
> 
> I am including the stack trace and my JDBCWordsDataSource with the additional
> debug code in it.
> 
> I am still configured to use HSQLDB which is reflected in the trace.
> 
> Matt Collier
> RemoteIT
> mco...@my...
> 877-4-NEW-LAN
> 
> 
> -----Original Message-----
> From: Nick Lothian <nl...@es...>
> To: Classifier4J <cla...@li...>
> Date: Wed, 12 Nov 2003 16:19:47 +1030
> Subject: RE: [Classifier4j-devel] Update Word Probability Break Down
> 
> > 
> > > ----
> > > More data on this issue:
> > > 
> > > Switching to HSQLDB produces the exact same results.  I have 
> > > attached the 
> > > revised connect.java. for use with HDSQLDB.
> > > ----
> > > Another interesting discovery.  If I attempt to run 
> > > connect.java a second time 
> > > immediately after running it the first time when in errors 
> > > out, the following 
> > > message is displayed immediately:
> > > 
> > > WordsDataSourceException Occurred : Problem creating table
> > > java.lang.IllegalArgumentException: IWordsDataSource can't be null
> > >    at net.sf.classifier4J.bayesian.BayesianClassifier.<init>
> > > (BayesianClassifier.java:141)
> > >    at net.sf.classifier4J.bayesian.BayesianClassifier.<init>
> > > (BayesianClassifier.java:128)
> > >    at net.sf.classifier4J.bayesian.BayesianClassifier.<init>
> > > (BayesianClassifier.java:118)
> > >    at Connect.main(Connect.java:26)
> > > Exception in thread "main"
> > > 
> > > However, if I wait about 60-90 seconds between executions, it 
> > > will process the 
> > > ~3900 records again and die.  
> > > ----
> > 
> > You are getting the second exception trace
> > (ava.lang.IllegalArgumentException: IWordsDataSource can't be null) because
> > you are ignoring the WordsDataSourceException, which means that the
> > IWordsDataSource you are using is null. That make sense.
> > 
> > Exactly why you are getting the original problem is escapign me at the
> > moment.
> > 
> > The error comes from line 247 in the CVS version of
> JDBCWordsDataSource.java
> > (you are using the CVS version, right?).
> > 
> > It occurs if an exception occurs somewhere in the following code:
> > 
> > 224                  			con =
> > connectionManager.getConnection();
> > 225                  			
> > 226                  			// check if the 
word_probability
> > table exists 
> > 227                  			DatabaseMetaData dbm =
> > con.getMetaData();		
> > 228                  			ResultSet rs = dbm.getTables
(null,
> > null, "WORD_PROBABILITY", null);
> > 229                  			if (!rs.next()) {
> > 230                  				// the table does not 
exist
> > 231                  				Statement stmt =
> > con.createStatement();	
> > 232                  				//	Under Axion 
1.0M1,
> > use 			
> > 233                  				//	 
stmt.executeUpdate(
> > "CREATE TABLE word_probability ( "
> > 234                  				//		
	+ "
> > word			VARCHAR(255) NOT NULL,"
> > 235                  				//		
	+ "
> > category		VARCHAR(20) NOT NULL,"
> > 236                  				//		
	+ "
> > match_count		INTEGER NOT NULL,"
> > 237                  				//		
	+ "
> > nonmatch_count	INTEGER NOT NULL, "
> > 238                  				//		
	+ "
> > PRIMARY KEY(word, category) ) ");				
> > 239                  				stmt.executeUpdate
( "CREATE
> > TABLE word_probability ( "
> > 240                  					+ " word
> > VARCHAR(255) NOT NULL,"
> > 241                  					+ " category
> > VARCHAR(20) NOT NULL,"
> > 242                  					+ " match_count
> > INT DEFAULT 0 NOT NULL,"
> > 243 			  					+ "
> > nonmatch_count	INT DEFAULT 0 NOT NULL, "
> > 244                  					+ " PRIMARY
> > KEY(word, category) ) ");				
> > 245                  			}
> > 
> > There are three possiblities here
> > 
> > 1) connectionManager.getConnection(); is failing
> > 2) DatabaseMetaData dbm = con.getMetaData(); or ResultSet rs =
> > dbm.getTables(null, null, "WORD_PROBABILITY", null); is failing
> > 3) The create table query is failing.
> > 
> > I suspect it is one of the first two. I found a reference to MySQL giving
> > incorrect error messages when tables are missing
> > <http://dbforums.com/arch/174/2003/10/952374>, and the error given is the
> > error you were getting when you were using MySQL.
> > 
> > Could you put an e.printStackStrace() in where it catches the SQLException
> > (ie, just before line 247) and send the stack trace you get?
> > 
> > Nick
> > 
> > 
> > 
> > 
> > -------------------------------------------------------
> > This SF.Net email sponsored by: ApacheCon 2003,
> > 16-19 November in Las Vegas. Learn firsthand the latest
> > developments in Apache, PHP, Perl, XML, Java, MySQL,
> > WebDAV, and more! http://www.apachecon.com/
> > _______________________________________________
> > Classifier4j-devel mailing list
> > Cla...@li...
> > https://lists.sourceforge.net/lists/listinfo/classifier4j-devel