From: Andrea Aime (JIRA) <jira@co...> - 2008-12-28 22:02:55
Postgis NG + large data set + non transactional read -> OOM
Issue Type: Bug
Components: data jdbc-ng
Affects Versions: 2.5.2
Reporter: Andrea Aime
Assignee: Justin Deoliveira
Fix For: 2.5.3
I was trying out the postgis NG datastore against the old one using the same old Texas roads database
when I found out that the NG one was systematically throwing OOM.
A little investigation shows that PostgreSQL can do partial fetches only if autocommit is false:
Looking into the old jdbc datastores code one can see that auto-commit is not set not even
for Transaction.AUTO_COMMIT. The upside is that partial fetches do work, the downside
is that apparently one has to use transactions to ensure changes are properly written?
Yet, the old postgis tests ensured that operations were working even on auto_commit,
not sure if it was an unintended side effect or if the transactions are committed on close unless
explicitly rolled back (I would find this odd thought)...
I made a few tests by adding setAutoCommit(false) to PostgisDialect.initializeConnection...
of course test failed. I had to add two things to get tests working again:
- add an explicit con.commit() in JDBCTestSetup.run()
- add an explicit con.commit() in PostgisDialect.postCreateTable
After those two gross hacks all tests are passing again but... ugh...
On the other side having a datastore that cannot load huge amounts of data
by streaming them is a major problem... :(
This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
Get latest updates about Open Source Projects, Conferences and News.