From: Richard G. <ric...@ya...> - 2005-11-14 15:01:22
|
Hi, I turned on the hibernate "show sql" option, and these are the sql statements it's trying to execute when it fails. If anyone has any ideas what's going wrong, I'd be grateful Thanks. Richard. Hibernate: select tm.id as id, tm.baseLocator as baseLoca2_ from topicmaps tm, locators locatordat0_ where (locatordat0_.notation=? and tm.baseLocator=locatordat0_.id)and(locatordat0_.address=? and tm.baseLocator=locatordat0_.id) Hibernate: select locator.id as id, locator.notation as notation, locator.address as address from locators locator where (locator.notation=? )and(locator.address=? ) Hibernate: select t.id as id, t.class as class, t.object_id as object_id, t.topicMap as topicMap, t.type as type, t.parent_association as parent_a6_, t.subject as subject, t.base_topic as base_topic, t.parent_topic as parent_t9_, t.data as data, t.dataLocator as dataLoc11_, t.container_id as contain12_, t.parent as parent from tmobjects t where (t.object_id=? )and(t.topicMap=? ) Hibernate: insert into locators (notation, address, id) values (?, ?, ?) Problem with checked-in Statement, discarding. java.lang.NullPointerException at oracle.jdbc.dbaccess.DBData.clearItem(DBData.java:431) at oracle.jdbc.dbaccess.DBDataSetImpl.clearItem(DBDataSetImpl.java:3528) at oracle.jdbc.driver.OraclePreparedStatement.clearParameters(OraclePreparedStatement.java:3401) at com.mchange.v2.c3p0.stmt.GooGooStatementCache.refreshStatement(GooGooStatementCache.java:452) at com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkinStatement(GooGooStatementCache.java:138) at com.mchange.v2.c3p0.impl.C3P0PooledConnection$1$WrapperStatementHelper.doClose(C3P0PooledConnection.java:429) at com.mchange.v2.c3p0.impl.C3P0PooledConnection$2.close(C3P0PooledConnection.java:476) at net.sf.hibernate.impl.BatcherImpl.closePreparedStatement(BatcherImpl.java:262) at net.sf.hibernate.impl.BatcherImpl.closeStatement(BatcherImpl.java:136) at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:125) at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:59) at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:56) at net.sf.hibernate.impl.BatcherImpl.prepareBatchStatement(BatcherImpl.java:109) at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:460) at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:442) at net.sf.hibernate.impl.ScheduledInsertion.execute(ScheduledInsertion.java:29) at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2382) at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2335) at net.sf.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1775) at net.sf.hibernate.impl.SessionImpl.getQueries(SessionImpl.java:1536) at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1501) at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1491) at org.tm4j.topicmap.hibernate.PersistenceWrapper.find(Unknown Source) at org.tm4j.topicmap.hibernate.TopicMapImpl.getObjectByID(Unknown Source) at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.checkID(Unknown Source) at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.save(Unknown Source) at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.createBaseName(Unknown Source) at org.tm4j.topicmap.hibernate.TopicImpl.createName(Unknown Source) at org.tm4j.topicmap.hibernate.TopicImpl.createName(Unknown Source) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:506) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:259) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:212) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:203) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:96) at mips.genre.topicmaps.Hibernate.main(Hibernate.java:62) org.tm4j.topicmap.TopicMapRuntimeException: Storage Exception. Cause: org.tm4j.topicmap.TopicMapRuntimeException: Storage exception. Cause: java.lang.RuntimeException: Internal inconsistency: A (not new) checking-out statement is not in deathmarch. Cause:Internal inconsistency: A (not new) checking-out statement is not in deathmarch. Cause:Storage exception. Cause: java.lang.RuntimeException: Internal inconsistency: A (not new) checking-out statement is not in deathmarch. Cause:Internal inconsistency: A (not new) checking-out statement is not in deathmarch. at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.save(Unknown Source) at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.createBaseName(Unknown Source) at org.tm4j.topicmap.hibernate.TopicImpl.createName(Unknown Source) at org.tm4j.topicmap.hibernate.TopicImpl.createName(Unknown Source) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:506) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:259) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:212) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:203) at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:96) at mips.genre.topicmaps.Hibernate.main(Hibernate.java:62) Caused by: org.tm4j.topicmap.TopicMapRuntimeException: Storage exception. Cause: java.lang.RuntimeException: Internal inconsistency: A (not new) checking-out statement is not in deathmarch. Cause:Internal inconsistency: A (not new) checking-out statement is not in deathmarch. at org.tm4j.topicmap.hibernate.PersistenceWrapper.find(Unknown Source) at org.tm4j.topicmap.hibernate.TopicMapImpl.getObjectByID(Unknown Source) at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.checkID(Unknown Source) ... 10 more Caused by: java.lang.RuntimeException: Internal inconsistency: A (not new) checking-out statement is not in deathmarch. at com.mchange.v2.c3p0.stmt.GooGooStatementCache.undeathmarchStatement(GooGooStatementCache.java:437) at com.mchange.v2.c3p0.stmt.GooGooStatementCache.removeStatement(GooGooStatementCache.java:273) at com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkinStatement(GooGooStatementCache.java:146) at com.mchange.v2.c3p0.impl.C3P0PooledConnection$1$WrapperStatementHelper.doClose(C3P0PooledConnection.java:429) at com.mchange.v2.c3p0.impl.C3P0PooledConnection$2.close(C3P0PooledConnection.java:476) at net.sf.hibernate.impl.BatcherImpl.closePreparedStatement(BatcherImpl.java:262) at net.sf.hibernate.impl.BatcherImpl.closeStatement(BatcherImpl.java:136) at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:125) at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:59) at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:56) at net.sf.hibernate.impl.BatcherImpl.prepareBatchStatement(BatcherImpl.java:109) at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:460) at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:442) at net.sf.hibernate.impl.ScheduledInsertion.execute(ScheduledInsertion.java:29) at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2382) at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2335) at net.sf.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1775) at net.sf.hibernate.impl.SessionImpl.getQueries(SessionImpl.java:1536) at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1501) at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1491) ... 13 more --- Richard Gregory <ric...@ya...> wrote: > Hi All, > > After a bit of a break I'm having another look at tm4j. I tried to use this approach of loading > an > entire existing topic map into my hibernate backend, using some of Daniel's code below, but I > get > the error below from the line > > tm.getFactory().copy(importedTM); > > If Kal or anyone else has any clues what is going wrong here I'd be grateful. > > Thanks, > > Richard. > > Heres the stack trace: > > Problem with checked-in Statement, discarding. > java.lang.NullPointerException > at oracle.jdbc.dbaccess.DBData.clearItem(DBData.java:431) > at oracle.jdbc.dbaccess.DBDataSetImpl.clearItem(DBDataSetImpl.java:3528) > at > oracle.jdbc.driver.OraclePreparedStatement.clearParameters(OraclePreparedStatement.java:3401) > at > com.mchange.v2.c3p0.stmt.GooGooStatementCache.refreshStatement(GooGooStatementCache.java:452) > at > com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkinStatement(GooGooStatementCache.java:138) > at > com.mchange.v2.c3p0.impl.C3P0PooledConnection$1$WrapperStatementHelper.doClose(C3P0PooledConnection.java:429) > at com.mchange.v2.c3p0.impl.C3P0PooledConnection$2.close(C3P0PooledConnection.java:476) > at net.sf.hibernate.impl.BatcherImpl.closePreparedStatement(BatcherImpl.java:262) > at net.sf.hibernate.impl.BatcherImpl.closeStatement(BatcherImpl.java:136) > at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:125) > at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:59) > at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:56) > at net.sf.hibernate.impl.BatcherImpl.prepareBatchStatement(BatcherImpl.java:109) > at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:460) > at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:442) > at net.sf.hibernate.impl.ScheduledInsertion.execute(ScheduledInsertion.java:29) > at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2382) > at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2335) > at net.sf.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1775) > at net.sf.hibernate.impl.SessionImpl.getQueries(SessionImpl.java:1536) > at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1501) > at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1491) > at org.tm4j.topicmap.hibernate.PersistenceWrapper.find(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicMapImpl.getObjectByID(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.checkID(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.save(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.createBaseName(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicImpl.createName(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicImpl.createName(Unknown Source) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:506) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:259) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:212) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:203) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:96) > at mips.genre.topicmaps.Hibernate.main(Hibernate.java:67) > org.tm4j.topicmap.TopicMapRuntimeException: Storage Exception. > Cause: org.tm4j.topicmap.TopicMapRuntimeException: Storage exception. > Cause: java.lang.RuntimeException: Internal inconsistency: A (not new) checking-out statement is > not in deathmarch. > Cause:Internal inconsistency: A (not new) checking-out statement is not in deathmarch. > Cause:Storage exception. > Cause: java.lang.RuntimeException: Internal inconsistency: A (not new) checking-out statement is > not in deathmarch. > Cause:Internal inconsistency: A (not new) checking-out statement is not in deathmarch. > at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.save(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.createBaseName(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicImpl.createName(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicImpl.createName(Unknown Source) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:506) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:259) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:212) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:203) > at org.tm4j.topicmap.TopicMapFactoryBase.copy(TopicMapFactoryBase.java:96) > at mips.genre.topicmaps.Hibernate.main(Hibernate.java:67) > Caused by: org.tm4j.topicmap.TopicMapRuntimeException: Storage exception. > Cause: java.lang.RuntimeException: Internal inconsistency: A (not new) checking-out statement is > not in deathmarch. > Cause:Internal inconsistency: A (not new) checking-out statement is not in deathmarch. > at org.tm4j.topicmap.hibernate.PersistenceWrapper.find(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicMapImpl.getObjectByID(Unknown Source) > at org.tm4j.topicmap.hibernate.TopicMapFactoryImpl.checkID(Unknown Source) > ... 10 more > Caused by: java.lang.RuntimeException: Internal inconsistency: A (not new) checking-out > statement > is not in deathmarch. > at > com.mchange.v2.c3p0.stmt.GooGooStatementCache.undeathmarchStatement(GooGooStatementCache.java:437) > at > com.mchange.v2.c3p0.stmt.GooGooStatementCache.removeStatement(GooGooStatementCache.java:273) > at > com.mchange.v2.c3p0.stmt.GooGooStatementCache.checkinStatement(GooGooStatementCache.java:146) > at > com.mchange.v2.c3p0.impl.C3P0PooledConnection$1$WrapperStatementHelper.doClose(C3P0PooledConnection.java:429) > at com.mchange.v2.c3p0.impl.C3P0PooledConnection$2.close(C3P0PooledConnection.java:476) > at net.sf.hibernate.impl.BatcherImpl.closePreparedStatement(BatcherImpl.java:262) > at net.sf.hibernate.impl.BatcherImpl.closeStatement(BatcherImpl.java:136) > at net.sf.hibernate.impl.BatcherImpl.executeBatch(BatcherImpl.java:125) > at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:59) > at net.sf.hibernate.impl.BatcherImpl.prepareStatement(BatcherImpl.java:56) > at net.sf.hibernate.impl.BatcherImpl.prepareBatchStatement(BatcherImpl.java:109) > at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:460) > at net.sf.hibernate.persister.EntityPersister.insert(EntityPersister.java:442) > at net.sf.hibernate.impl.ScheduledInsertion.execute(ScheduledInsertion.java:29) > at net.sf.hibernate.impl.SessionImpl.executeAll(SessionImpl.java:2382) > at net.sf.hibernate.impl.SessionImpl.execute(SessionImpl.java:2335) > at net.sf.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:1775) > at net.sf.hibernate.impl.SessionImpl.getQueries(SessionImpl.java:1536) > at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1501) > at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1491) > ... 13 more > > > > --- Daniel Meier <mei...@hi...> wrote: > > > > > Hello Kal > > > > Thx for your answer. As you said, I reimplemented the import. > > First I create a topic map in memory, then I copy the topic map > > to the database. This is working fine, expect the topic map has > > scopes defined. I tried the tmworld.xtm topic map and I get an > > java.lang.ClassCastException at > > org.tm4j.topicmap.hibernate.ScopedObjectImpl.setScope(ScopedObjectImpl.java: > > 223) > > > > Do you have an idea why this happens? I use tm4j 0.9.7src. > > Hope you can help me...thx in advance... > > > > My code looks like: > > > > public class copyToDB { > > public static void main(String[] args) { > > new copyToDB(args); > > } > > public copyToDB(String[] args) { > > File f = new File("src/tm/tmworld.xtm"); > > try { > > TopicMap importedTM = loadXtmFromFile(f); > > > > // Hibernate > > TopicMapProviderFactory tmpf = new > > org.tm4j.topicmap.hibernate.TopicMapProviderFactoryImpl(); > > Properties providerProps = new Properties(); > > providerProps.load(new > > FileInputStream("src/tm/provider.properties")); > > TopicMapProvider provider = > > tmpf.newTopicMapProvider(providerProps); > > > > > > ProviderTransaction txn = > > provider.openTransaction(); > > > > Locator base = > > provider.getLocatorFactory().createLocator("URI", f.toURL().toString()); > > TopicMap tm = provider.createTopicMap(base); > > tm.getFactory().copy(importedTM); > > > > txn.commit(); > > > > } catch(Exception e) { > > e.printStackTrace(); > > } > > } > > > > public static TopicMap loadXtmFromFile(File f) throws Exception { > > if (!f.exists()) { > > throw new FileNotFoundException("Could not locate the > > specified input file."); > > } > > TopicMapProviderFactory m_providerFactory = new > > org.tm4j.topicmap.memory.TopicMapProviderFactoryImpl(); > > TopicMapProvider m_provider = > > m_providerFactory.newTopicMapProvider(System.getProperties()); > > Locator baseLocator = > > m_provider.getLocatorFactory().createLocator("URI", f.toURL().toString()); > > TopicMapSource src = new SerializedTopicMapSource(new > > FileInputStream(f), baseLocator); > > > > return m_provider.addTopicMap(src); > > } > > } > > > > > > > > Hi Daniel, > > > > Yeah its slow, and partially it is due to the frequency of commits that > > Hibernate makes, but there is also a more tricky fundamental problem. If you > > are importing an XTM file, there is no guarantee that the XTM is properly > > merged. That means that as you add topics you need to check to see if they > > should merge - which in turn means you need to check the subject indicators, > > source locators and scoped names of the topics that you have imported up > > until now. So the XTM import code uses the indexes to do those lookups. Of > > course, with the Hibernate implementation the indexes are implemented as > > queries - so that means that you do need to commit at least after every > > topic is added for merging to work properly. > > > > So if you have a really large TM, or if you are importing an XTM file into > > an existing topic map in the database (i.e. merging a file into an existing > > TM), then I think this is always going to be a problem > > > > However, your case is slightly different (and probably quite common) - you > > have a relatively large XTM file, but it is not so large that it could not > > all be processed for merging in memory first (you might a big chunk of > > memory assigned to the VM, but it should be doable). So perhaps you could > > make a tuned import function for the Hibernate backend that does an import > > into a temporary in-memory backend first, applies all the merging, and then > > writes the objects to Hibernate in a single transaction. My feeling is that > > you should be able to do all of this using the existing APIs - we added a > > feature for transaction control on the backends in the last version. > > > > In the long term, I think the fix for this is to be smarter about the import > > into the database and to have a separate importer for Hibernate backends > > that works with the database instead of against it as we have at the moment. > > But perhaps for your short term needs doing the two-step XTM -> Memory -> > > Hibernate import might work? > > > > Cheers, > > > > Kal > > > > > -----Original Message----- > > > From: tm4...@li... [mailto:tm4j-users- > > > ad...@li...] On Behalf Of Daniel Meier > > > Sent: 12 October 2005 19:21 > > > To: tm4...@li... > > > Subject: [TM4J-users] importing xtm to Db with hibernate > > > > > > Hello > > > > > > > > > > > > Im trying to import a "big" xtm file (~6MByte, ~4000 topics and ~ 4000 > > > assocciations ) to an oracle 8 database (hibernate backend). > > > > > > Code looks like: > > > > > > > > > > > > TopicMapProviderFactory factory = TopicMapProviderFactory.newInstance(); > > > > > > TopicMapProvider provider = factory.newTopicMapProvider(); > > > > > > TopicMapSource source = new > > > SerializedTopicMapSource("c:\\temp\\opera.xtm"); > > > > > > TopicMap map = provider.addTopicMap(source); > > > > > > > > > > > > > > > > > > Now this process takes very looong (>1h). When I log the hibernate sql > > > commands, > > > > > > I see that the backend is autocommiting every insert/update. This is in my > > > mind > > > > > > not good, and is probably the problem. One commit at the end is enough > > > during an import. > > > > > > So I set autocommit in the provider.properties to false, and added the > > > code: > > > > > > . > > > > > > TopicMap map = provider.addTopicMap(source); > > > > > > Provider.getOpenTransaction().commit(); > > > > > > > > > > > > But this doesen't helped. > > > > > > Maybe the problem is that provider.isTransactional() returns false. I > > > think the oracle DB > > > > > > should be able to handle transactions. (I use the jdbc driver from > > > oracle). > > > > > > > > > > > > What can I do? > > > > > > > > > > > > Greetings, Daniel > > > > > > > > > > > > > > > > ------------------------------------------------------- > > This SF.Net email is sponsored by: > > Power Architecture Resource Center: Free content, downloads, discussions, > > and more. http://solutions.newsforge.com/ibmarch.tmpl > > _______________________________________________ > > Tm4j-users mailing list > > Tm4...@li... > > https://lists.sourceforge.net/lists/listinfo/tm4j-users > > > > > > > > > > ___________________________________________________________ > Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail > http://uk.messenger.yahoo.com > ___________________________________________________________ How much free photo storage do you get? Store your holiday snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com |