RE: [Classifier4j-devel] Update Word Probability Break Down

[Nick L. <nl...@es...>]

> ----
> More data on this issue:
> 
> Switching to HSQLDB produces the exact same results.  I have 
> attached the 
> revised connect.java. for use with HDSQLDB.
> ----
> Another interesting discovery.  If I attempt to run 
> connect.java a second time 
> immediately after running it the first time when in errors 
> out, the following 
> message is displayed immediately:
> 
> WordsDataSourceException Occurred : Problem creating table
> java.lang.IllegalArgumentException: IWordsDataSource can't be null
>    at net.sf.classifier4J.bayesian.BayesianClassifier.<init>
> (BayesianClassifier.java:141)
>    at net.sf.classifier4J.bayesian.BayesianClassifier.<init>
> (BayesianClassifier.java:128)
>    at net.sf.classifier4J.bayesian.BayesianClassifier.<init>
> (BayesianClassifier.java:118)
>    at Connect.main(Connect.java:26)
> Exception in thread "main"
> 
> However, if I wait about 60-90 seconds between executions, it 
> will process the 
> ~3900 records again and die.  
> ----

You are getting the second exception trace
(ava.lang.IllegalArgumentException: IWordsDataSource can't be null) because
you are ignoring the WordsDataSourceException, which means that the
IWordsDataSource you are using is null. That make sense.

Exactly why you are getting the original problem is escapign me at the
moment.

The error comes from line 247 in the CVS version of JDBCWordsDataSource.java
(you are using the CVS version, right?).

It occurs if an exception occurs somewhere in the following code:

224                  			con =
connectionManager.getConnection();
225                  			
226                  			// check if the word_probability
table exists 
227                  			DatabaseMetaData dbm =
con.getMetaData();		
228                  			ResultSet rs = dbm.getTables(null,
null, "WORD_PROBABILITY", null);
229                  			if (!rs.next()) {
230                  				// the table does not exist
231                  				Statement stmt =
con.createStatement();	
232                  				//	Under Axion 1.0M1,
use 			
233                  				//	 stmt.executeUpdate(
"CREATE TABLE word_probability ( "
234                  				//			+ "
word			VARCHAR(255) NOT NULL,"
235                  				//			+ "
category		VARCHAR(20) NOT NULL,"
236                  				//			+ "
match_count		INTEGER NOT NULL,"
237                  				//			+ "
nonmatch_count	INTEGER NOT NULL, "
238                  				//			+ "
PRIMARY KEY(word, category) ) ");				
239                  				stmt.executeUpdate( "CREATE
TABLE word_probability ( "
240                  					+ " word
VARCHAR(255) NOT NULL,"
241                  					+ " category
VARCHAR(20) NOT NULL,"
242                  					+ " match_count
INT DEFAULT 0 NOT NULL,"
243 			  					+ "
nonmatch_count	INT DEFAULT 0 NOT NULL, "
244                  					+ " PRIMARY
KEY(word, category) ) ");				
245                  			}

There are three possiblities here

1) connectionManager.getConnection(); is failing
2) DatabaseMetaData dbm = con.getMetaData(); or ResultSet rs =
dbm.getTables(null, null, "WORD_PROBABILITY", null); is failing
3) The create table query is failing.

I suspect it is one of the first two. I found a reference to MySQL giving
incorrect error messages when tables are missing
<http://dbforums.com/arch/174/2003/10/952374>, and the error given is the
error you were getting when you were using MySQL.

Could you put an e.printStackStrace() in where it catches the SQLException
(ie, just before line 247) and send the stack trace you get?

Nick