Re: [GeoNetwork-devel] ticket #163: Simultaneous reharvest fail

[<Sim...@cs...>]

Hi Doug and others,

Which database are you using when you get this bug?

I have also come across the concurrent serializable transaction conflict in the recent bug on Dbms retries and before that I used to get it in lots of situations with ORACLE (eg. doing a big batch import etc).

I think the problem is in part due to the rather restrictive (but very safe) way in which access to data in the database is synchronized between different threads. At the moment (except for ORACLE) we have exclusive locks ie. one writer or one reader can have a lock at any time (isolation level SERIALIZABLE). This appears to be in part due to McKoi which doesn't support any other isolation level and because this is the safest way (in terms of consistency) to handle this issue.

We could change the default isolation level for all databases except McKoi to allow data to be read at any time as long as there is no write in progress (isolation level READ_COMMITTED - for all except McKoi which shouldn't be used in production anyway) which would help with this problem and also improve performance too. However we should check our database interaction to see whether there are any issues.

The other issue with the harvest is that by my reckoning, each harvest process uses one connection to the database - if you attempt to start 35 at the same time by selecting them all and choosing run then I think your harvest will fail anyway - you need to schedule them in groups according to the database pool size for your database (see <resources><resource enabled="true">..... in web/geonetwork/WEB-INF/config.xml) or more simply, just increase the poolsize - default poolsize is 10.

Cheers,
Simon
________________________________________
From: GeoNetwork opensource Developer website [tra...@os...]
Sent: Friday, 23 October 2009 1:57 AM
Subject: Re: [GeoNetwork-devel] [GeoNetwork opensource Developer website] #163: Simultaneous reharvest fail

#163: Simultaneous reharvest fail
-----------------------+----------------------------------------------------
  Reporter:  ddnebert  |       Owner:  geo...@li...
      Type:  defect    |      Status:  new
  Priority:  major     |   Milestone:
 Component:  General   |     Version:
Resolution:            |    Keywords:  harvest, concurrency, run
-----------------------+----------------------------------------------------
Comment (by ddnebert):

 Class: MSQL Exception, Error: Concurrent Serializable Transaction Conflict

--
Ticket URL: <http://trac.osgeo.org/geonetwork/ticket/163#comment:2>
GeoNetwork opensource Developer website <http://trac.osgeo.org/geonetwork>
GeoNetwork opensource is a standards based, Free and Open Source catalog application to manage spatially referenced resources through the web. It provides powerful metadata editing and search functions as well as an embedded interactive web map viewer. This website contains information related to the development of the software.
------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
GeoNetwork-devel mailing list
Geo...@li...
https://lists.sourceforge.net/lists/listinfo/geonetwork-devel
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork