Re: [Bigdata-developers] Design patterns for shutdown of embedded services and in particular for the Journal.

[Bryan T. <br...@sy...>]

Martyn points out that this may have been an uninterruptable Lock.lock() invocation in JournalTransactionService.SinglePhaseCommit.  I think that a survey of the code would discover many places where uninteruptable lock acquisitions are attempted.

Bryan

> -----Original Message-----
> From: Bryan Thompson [mailto:br...@sy...] 
> Sent: Monday, April 12, 2010 10:38 AM
> To: big...@li...
> Subject: [Bigdata-developers] Design patterns for shutdown of 
> embedded services and in particular for the Journal.
> 
> We currently have a deadlock which occasionally arises in 
> Journal#shutdownNow().  That method closes down various 
> services used by the journal (an executor service, the 
> concurrency manager (which also shuts down the write 
> service), and the local transaction manager).  If there is a 
> concurrent commit running, then the code will occasionally 
> deadlock as illustrated below.  
> 
> I'd like to find someone with experience in shutdown patterns 
> for embedded services who could help out on this shutdown 
> pattern.  Any volunteers?
> 
> Bryan
> 
> "pool-1-thread-560" daemon prio=10 tid=0x52e9ac00 nid=0x7ae6 
> waiting on condition [0x41869000]
>    java.lang.Thread.State: WAITING (parking)
> 	at sun.misc.Unsafe.park(Native Method)
> 	- parking to wait for  <0x88a1fe40> (a 
> java.util.concurrent.FutureTask$Sync)
> 	at 
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> 	at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndC
> heckInterrupt(AbstractQueuedSynchronizer.java:747)
> 	at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquir
> eSharedInterruptibly(AbstractQueuedSynchronizer.java:905)
> 	at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireS
> haredInterruptibly(AbstractQueuedSynchronizer.java:1217)
> 	at 
> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:218)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> 	at 
> com.bigdata.journal.JournalTransactionService.commitImpl(Journ
> alTransactionService.java:300)
> 	at 
> com.bigdata.service.AbstractTransactionService.commit(Abstract
> TransactionService.java:1626)
> 	at com.bigdata.journal.Journal.commit(Journal.java:726)
> 
> "main" prio=10 tid=0x08f62c00 nid=0x6c29 waiting on condition 
> [0xb7395000]
>    java.lang.Thread.State: WAITING (parking)
> 	at sun.misc.Unsafe.park(Native Method)
> 	- parking to wait for  <0x88a241b8> (a 
> java.util.concurrent.locks.ReentrantLock$NonfairSync)
> 	at 
> java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
> 	at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndC
> heckInterrupt(AbstractQueuedSynchronizer.java:747)
> 	at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQ
> ueued(AbstractQueuedSynchronizer.java:778)
> 	at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(
> AbstractQueuedSynchronizer.java:1114)
> 	at 
> java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(Reen
> trantLock.java:186)
> 	at 
> java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262)
> 	at 
> com.bigdata.service.AbstractTransactionService.shutdownNow(Abs
> tractTransactionService.java:488)
> 	at com.bigdata.journal.Journal$1.shutdownNow(Journal.java:201)
> 	at com.bigdata.journal.Journal.shutdownNow(Journal.java:852)
> 	- locked <0x86a5dc40> (a com.bigdata.journal.Journal)
> --------------------------------------------------------------
> ----------------
> Download Intel&#174; Parallel Studio Eval Try the new 
> software tools for yourself. Speed compiling, find bugs 
> proactively, and fine-tune applications for parallel performance.
> See why Intel Parallel Studio got high marks during beta.
> http://p.sf.net/sfu/intel-sw-dev
> _______________________________________________
> Bigdata-developers mailing list
> Big...@li...
> https://lists.sourceforge.net/lists/listinfo/bigdata-developers
>