[ jEdit-devel ] [ jedit-Plugin Bugs-3285504 ] Lucene + ProjectViewer: Deadlock on re-import

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Plugin Bugs item #3285504, was opened at 2011-04-12 13:39
Message generated for change (Comment added) made by ezust
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=565475&aid=3285504&group_id=588

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Closed
Resolution: Fixed
Priority: 5
Private: No
Submitted By: Alan Ezust (ezust)
Assigned to: Nobody/Anonymous (nobody)
Summary: Lucene + ProjectViewer: Deadlock on re-import

Initial Comment:
I have a particular project called "Qt4-src".
It is currently  the Qt-4.7.1-everywhere-opensource.src.tar.gzl unzipped. 
It is indexed in lucene. Sometimes I change its root directory from one place to another. 
for example, when I upgrade to a newer version of Qt.

Whenever I do that, ProjectViewer asks me "remove old files from project root?"
I say yes. Then it greys out, I see "re-importing" and then jEdit crashes.

I re-run jEdit and try the same re-import. crash again.

I have to delete the project. Lucene then asks me if I want to delete the index.
Then I can re-create the project, and re-index it in lucene. Everyone is happy.

I shouldn't have to do this every time though.

----------------------------------------------------------------------

>Comment By: Alan Ezust (ezust)
Date: 2011-06-06 20:37

Message:
Ahhh, much better!!! Thanks!

----------------------------------------------------------------------

Comment By: Marcelo Vanzin (vanza)
Date: 2011-06-05 21:41

Message:
Should be much better with commit 2f425c0.

----------------------------------------------------------------------

Comment By: Marcelo Vanzin (vanza)
Date: 2011-06-05 19:00

Message:
There's no file being written to on the PV side, it's just updating its UI.
I don't know if Lucene is doing anything, but it doesn't show up in the
thread dump, so I'm assuming it's not doing anything.

----------------------------------------------------------------------

Comment By: Alan Ezust (ezust)
Date: 2011-06-05 18:54

Message:
If a task is running shouldn't I be able to see progress somehow in the
size of a file gradually increasing? Because I don't, in the projectviewer
projects or the lucene index area.  There is no progress showing in the
task monitor. 
Also, it doesn't take very long (more than a minute or two) to import and
reindex this project from scratch. If I wait 10 minutes and nothing happens
on a reimport, and I can't use the editor during that time, that's not
good. 

----------------------------------------------------------------------

Comment By: Marcelo Vanzin (vanza)
Date: 2011-06-05 18:43

Message:
Yes, the GUI is unresponsive, there's nothing in the activity log, and the
CPU is pegged because there is no deadlock and there is a task still
running. If you wait long enough it will finish.

There's no deadlock here. Maybe the code can be changed to let the UI be
more responsive, but you won't see lower CPU utilization nor decrease in
the time it takes to make the PV UI re-enable itself. You just have a
really large project, it takes a while to update a JTree with that many
nodes.

----------------------------------------------------------------------

Comment By: Alan Ezust (ezust)
Date: 2011-06-05 18:33

Message:
I think it is a deadlock because:
  1. Nothing is being appended to the activity log
  2. projectviewer combobox is greyed out
  3. no progress indication going on anywhere
  4. no GUI events are being processed.
One of my CPUs is stuck at 100% as a result of this, so there is a busy
loop consuming CPU at this time.

And there is nothing i can do but kill the process at this point. If
that's not a deadlock, I don't know what is...

----------------------------------------------------------------------

Comment By: Marcelo Vanzin (vanza)
Date: 2011-06-05 17:01

Message:
Again, why do you think there's a deadlock? The thread dump you posted does
not show any deadlock at all. It shows a UI task still running, and an I/O
task waiting for that UI task to finish. That's normal. Since your project
is large it may just be taking a long time. Depending on how the Lucene
plugin interacts with PV, it may take longer than usual.

----------------------------------------------------------------------

Comment By: Alan Ezust (ezust)
Date: 2011-06-05 15:27

Message:
Can still reproduce after increasing max heap size. thread dump attached.

----------------------------------------------------------------------

Comment By: Alan Ezust (ezust)
Date: 2011-06-05 12:15

Message:
s/crash/deadlock in my initial post, that was a mistake. I never saw
crashing behavior.

----------------------------------------------------------------------

Comment By: Eric Le Lay (kerik-sf)
Date: 2011-06-05 12:06

Message:
I also observed a deadlock some time ago (with another plugin accessing
PluginManager in some way, which caused the deadlock).
I'll try to find the app image from back then.

----------------------------------------------------------------------

Comment By: Marcelo Vanzin (vanza)
Date: 2011-06-04 15:43

Message:
Deadlocks don't cause crashes... also your thread dump doesn't show any
deadlocks that I can see. There's a PV I/O task waiting on the AWT thread,
but the AWT thead is not blocked, it's still running, executing the task
the I/O thread is waiting on.

Are you sure you're not just running out of memory? The VM should throw an
error instead of crashing, but you never know. A heap dump could help if
it's really a case of running out of memory.

----------------------------------------------------------------------

Comment By: Alan Ezust (ezust)
Date: 2011-04-12 14:29

Message:
activity log attached.

----------------------------------------------------------------------

Comment By: Alan Ezust (ezust)
Date: 2011-04-12 14:24

Message:
Here is the thread dump . Deadlock possible?

[ezust@cerberus] /home/ezust/Trolltech> 2011-04-12 14:22:30
Full thread dump Java HotSpot(TM) 64-Bit Server VM (19.1-b02 mixed mode):

"CC::Worker #1" daemon prio=10 tid=0x00007f11dc9bc800 nid=0x5b56 in
Object.wait() [0x00007f11d1fd4000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:485)
        at java.awt.EventQueue.invokeAndWait(EventQueue.java:1038)
        - locked <0x00000000f9a2b558> (a
java.awt.EventQueue$1AWTInvocationLock)
        at
javax.swing.SwingUtilities.invokeAndWait(SwingUtilities.java:1326)
        at projectviewer.PVActions.swingInvoke(PVActions.java:338)
        at projectviewer.importer.Importer.run(Importer.java:545)
        at common.threads.WorkRequest.run(WorkRequest.java:80)
        at
common.threads.WorkerThreadPool$WorkerThread.run(WorkerThreadPool.java:235)

"jEdit server daemon [/home/ezust/.jedit/server]" daemon prio=10
tid=0x00007f11d4210000 nid=0x5b50 runnable [0x00007f11d22d7000]
   java.lang.Thread.State: RUNNABLE
        at java.net.PlainSocketImpl.socketAccept(Native Method)
        at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
        - locked <0x00000000c2364090> (a java.net.SocksSocketImpl)
        at java.net.ServerSocket.implAccept(ServerSocket.java:462)
        at java.net.ServerSocket.accept(ServerSocket.java:430)
        at org.gjt.sp.jedit.EditServer.run(EditServer.java:132)

"jEdit I/O #4" prio=10 tid=0x00007f11d4062800 nid=0x5b4f in Object.wait()
[0x00007f11d23d8000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000c2621238> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:485)
        at org.gjt.sp.util.WorkThread.doRequests(WorkThread.java:201)
        - locked <0x00000000c2621238> (a java.lang.Object)
        at org.gjt.sp.util.WorkThread.run(WorkThread.java:157)

"jEdit I/O #3" prio=10 tid=0x00007f11d4062000 nid=0x5b4e in Object.wait()
[0x00007f11d24d9000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000c2621238> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:485)
        at org.gjt.sp.util.WorkThread.doRequests(WorkThread.java:201)
        - locked <0x00000000c2621238> (a java.lang.Object)
        at org.gjt.sp.util.WorkThread.run(WorkThread.java:157)

"jEdit I/O #2" prio=10 tid=0x00007f11d4074000 nid=0x5b4d in Object.wait()
[0x00007f11d25da000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000c2621238> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:485)
        at org.gjt.sp.util.WorkThread.doRequests(WorkThread.java:201)
        - locked <0x00000000c2621238> (a java.lang.Object)
        at org.gjt.sp.util.WorkThread.run(WorkThread.java:157)

"jEdit I/O #1" prio=10 tid=0x00007f11d4026000 nid=0x5b4c in Object.wait()
[0x00007f11d26db000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000c2621238> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:485)
        at org.gjt.sp.util.WorkThread.doRequests(WorkThread.java:201)
        - locked <0x00000000c2621238> (a java.lang.Object)
        at org.gjt.sp.util.WorkThread.run(WorkThread.java:157)

"pool-1-thread-1" prio=10 tid=0x0000000041b2b800 nid=0x5b4b waiting on
condition [0x00007f11d27dc000]
   java.lang.Thread.State: WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000c33becc8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
        at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987)
        at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
        at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
        at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
        at java.lang.Thread.run(Thread.java:662)

"DestroyJavaVM" prio=10 tid=0x0000000040d96000 nid=0x5b35 waiting on
condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"TimerQueue" daemon prio=10 tid=0x000000004174f800 nid=0x5b4a in
Object.wait() [0x00007f11db7d6000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at javax.swing.TimerQueue.run(TimerQueue.java:232)
        - locked <0x00000000c273ff98> (a javax.swing.TimerQueue)
        at java.lang.Thread.run(Thread.java:662)

"AWT-EventQueue-0" prio=10 tid=0x0000000040fc5000 nid=0x5b46 runnable
[0x00007f11db9d7000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.StringBuilder.append(StringBuilder.java:119)
        at projectviewer.vpt.VPTFile.toString(VPTFile.java:272)
        at javax.swing.JTree.convertValueToText(JTree.java:1563)
        at
javax.swing.tree.DefaultTreeCellRenderer.getTreeCellRendererComponent(DefaultTreeCellRenderer.java:346)
        at
projectviewer.vpt.VPTCellRenderer.getTreeCellRendererComponent(VPTCellRenderer.java:92)
        at
javax.swing.plaf.basic.BasicTreeUI$NodeDimensionsHandler.getNodeDimensions(BasicTreeUI.java:2712)
        at
javax.swing.tree.AbstractLayoutCache.getNodeDimensions(AbstractLayoutCache.java:475)
        at
javax.swing.tree.VariableHeightLayoutCache$TreeStateNode.updatePreferredSize(VariableHeightLayoutCache.java:1342)
        at
javax.swing.tree.VariableHeightLayoutCache$TreeStateNode.expand(VariableHeightLayoutCache.java:1478)
        at
javax.swing.tree.VariableHeightLayoutCache$TreeStateNode.expand(VariableHeightLayoutCache.java:1270)
        at
javax.swing.tree.VariableHeightLayoutCache.rebuild(VariableHeightLayoutCache.java:725)
        at
javax.swing.tree.VariableHeightLayoutCache.treeStructureChanged(VariableHeightLayoutCache.java:626)
        at
javax.swing.plaf.basic.BasicTreeUI$Handler.treeStructureChanged(BasicTreeUI.java:3824)
        at
javax.swing.tree.DefaultTreeModel.fireTreeStructureChanged(DefaultTreeModel.java:561)
        at
javax.swing.tree.DefaultTreeModel.nodeStructureChanged(DefaultTreeModel.java:347)
        at
projectviewer.vpt.ProjectCustomTreeModel.nodeStructureChanged(ProjectCustomTreeModel.java:171)
        at
projectviewer.vpt.ProjectTreePanel.nodesWereInserted(ProjectTreePanel.java:355)
        at
projectviewer.ProjectViewer.insertNodeInto(ProjectViewer.java:171)
        at
projectviewer.importer.Importer.processAddedNodes(Importer.java:677)
        at projectviewer.importer.Importer.access$000(Importer.java:77)
        at projectviewer.importer.Importer$2.run(Importer.java:548)
        at
java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:199)
        at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:642)
        at java.awt.EventQueue.access$000(EventQueue.java:85)
        at java.awt.EventQueue$1.run(EventQueue.java:603)
        at java.awt.EventQueue$1.run(EventQueue.java:601)
        at java.security.AccessController.doPrivileged(Native Method)
        at
java.security.AccessControlContext$1.doIntersectionPrivilege(AccessControlContext.java:87)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:612)
        at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
        at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
        at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
        at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
        at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
        at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)

"AWT-Shutdown" prio=10 tid=0x00000000411eb000 nid=0x5b45 in Object.wait()
[0x00007f11dbad9000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        - waiting on <0x00000000c274b2a0> (a java.lang.Object)
        at java.lang.Object.wait(Object.java:485)
        at sun.awt.AWTAutoShutdown.run(AWTAutoShutdown.java:265)
        - locked <0x00000000c274b2a0> (a java.lang.Object)
        at java.lang.Thread.run(Thread.java:662)

"AWT-XAWT" daemon prio=10 tid=0x00007f11dc2fe800 nid=0x5b43 runnable
[0x00007f11e0160000]
   java.lang.Thread.State: RUNNABLE
        at sun.awt.X11.XToolkit.waitForEvents(Native Method)
        at sun.awt.X11.XToolkit.run(XToolkit.java:579)
        at sun.awt.X11.XToolkit.run(XToolkit.java:543)
        at java.lang.Thread.run(Thread.java:662)

"Java2D Disposer" daemon prio=10 tid=0x00007f11dc104800 nid=0x5b42 in
Object.wait() [0x00007f11e0764000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0x00000000c2efd1c8> (a
java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at sun.java2d.Disposer.run(Disposer.java:127)
        at java.lang.Thread.run(Thread.java:662)

"Low Memory Detector" daemon prio=10 tid=0x0000000040e4d000 nid=0x5b40
runnable [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=10 tid=0x0000000040e4b000 nid=0x5b3f waiting
on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=10 tid=0x0000000040e48000 nid=0x5b3e waiting
on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=10 tid=0x0000000040e46000 nid=0x5b3d
waiting on condition [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=10 tid=0x0000000040e29000 nid=0x5b3c in
Object.wait() [0x00007f11e2371000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
        - locked <0x00000000c2326248> (a
java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
        at
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=10 tid=0x0000000040e21800 nid=0x5b3b in
Object.wait() [0x00007f11e2472000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(Native Method)
        at java.lang.Object.wait(Object.java:485)
        at
java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
        - locked <0x00000000c23261e0> (a java.lang.ref.Reference$Lock)

"VM Thread" prio=10 tid=0x0000000040e1a800 nid=0x5b3a runnable 

"GC task thread#0 (ParallelGC)" prio=10 tid=0x0000000040da9000 nid=0x5b36
runnable 

"GC task thread#1 (ParallelGC)" prio=10 tid=0x0000000040dab000 nid=0x5b37
runnable 

"GC task thread#2 (ParallelGC)" prio=10 tid=0x0000000040dac800 nid=0x5b38
runnable 

"GC task thread#3 (ParallelGC)" prio=10 tid=0x0000000040dae800 nid=0x5b39
runnable 

"VM Periodic Task Thread" prio=10 tid=0x0000000040e58000 nid=0x5b41
waiting on condition 

JNI global references: 1682

Heap
 PSYoungGen      total 254336K, used 239160K [0x00000000eb560000,
0x0000000100000000, 0x0000000100000000)
  eden space 225536K, 96% used
[0x00000000eb560000,0x00000000f89700c8,0x00000000f91a0000)
  from space 28800K, 76% used
[0x00000000f91a0000,0x00000000fa71e048,0x00000000fadc0000)
  to   space 31232K, 0% used
[0x00000000fe180000,0x00000000fe180000,0x0000000100000000)
 PSOldGen        total 102528K, used 60924K [0x00000000c2000000,
0x00000000c8420000, 0x00000000eb560000)
  object space 102528K, 59% used
[0x00000000c2000000,0x00000000c5b7f098,0x00000000c8420000)
 PSPermGen       total 75456K, used 33788K [0x00000000bce00000,
0x00000000c17b0000, 0x00000000c2000000)
  object space 75456K, 44% used
[0x00000000bce00000,0x00000000beeff238,0x00000000c17b0000)

----------------------------------------------------------------------

Comment By: Matthieu Casanova (kpouer)
Date: 2011-04-12 14:05

Message:
You don't have anything interesting in the activity.log ?

----------------------------------------------------------------------

Comment By: Alan Ezust (ezust)
Date: 2011-04-12 13:51

Message:
I tried to reproduce this with a smaller project and it does not cause a
crash. It seems to need to be a large project.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=565475&aid=3285504&group_id=588

[ jEdit-devel ] [ jedit-Plugin Bugs-3285504 ] Lucene + ProjectViewer: Deadlock on re-import

jEdit is a programmer's text editor written in Java.

[ jEdit-devel ] [ jedit-Plugin Bugs-3285504 ] Lucene + ProjectViewer: Deadlock on re-import