Menu
▾
▴
[Artifactory-users] Yum indexing deadlocked on PipedInputStream
|
From: Ryan M. <rya...@gm...> - 2012-08-29 20:06:25
|
I've encountered issues with YUM metadata indexing in Artifactory Pro 2.6.2. It appears that there's a deadlock between two threads which cause indexing of all YUM repositories to stop. It also causes Artifactory to enter a wait(1000) loop which is consuming 85%-95% of a CPU core and affecting performance of the service. "pool-24-thread-1" running org.artifactory.addon.yum.YumAddonImpl$1.produce(Artifactory:285) "art-exec-197" running: org.artifactory.addon.yum.YumAddonImpl.executeAsyncRepositoryYumMetadataCalculation(Artifactory:154) I'm especially curious about the "art-exec-197" thread which contains both a PipedOutputStream.write and a PipedInputStream.receive. According to the JavaDoc for class PipedInputStream [1]: "Typically, data is read from a PipedInputStream object by one thread and data is written to the corresponding PipedOutputStream by some other thread. Attempting to use both objects from a single thread is not recommended, as it may deadlock the thread." I've included [2] a thread dump from "kill -3" which shows that both threads are locked on <0x000000077ed6f140> (a com.gc.iotools.stream.is.InputStreamFromOutputStream$MyPipedInputStream). Both of these threads are stuck in java.io.PipedInputStream.awaitSpace which according to the Java Docs is essentially running a wait(1000); notifyAll(); and this is hogging an entire CPU core. My Artifactory instance contains three repositories configured with "Yum Calculation" enabled. Each of these generates metadata at depth "3" for RPMs uploaded by Jenkins. I have several hundred RPMs across these three repositories and the many subfolders, and when YUM indexing stops it *KILLS* my deployment pipeline, which is my primary use case for Artifactory. My system information is included [3]. Please advise on a workaround in the short term, and I would really appreciate if this could be addressed in a subsequent release. This is currently locking up more than once a day, and it's impacting my entire organization. ~Ryan ---- [1] Class PipedInputStream JavaDoc: http://docs.oracle.com/javase/6/docs/api/java/io/PipedInputStream.html [2] Thread dump: Full thread dump Java HotSpot(TM) 64-Bit Server VM (20.8-b03 mixed mode): "art-exec-264" prio=10 tid=0x00002aaac13a9800 nid=0x3f76 waiting on condition [0x0000000040c19000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000075ac77d80> (a java.util.concurrent.SynchronousQueue$TransferStack) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:424) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:323) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:874) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:945) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:662) "Thread-2385" daemon prio=10 tid=0x00002aaab8855000 nid=0x3f47 runnable [0x0000000044840000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read1(BufferedInputStream.java:258) at java.io.BufferedInputStream.read(BufferedInputStream.java:317) - locked <0x00000007582920c8> (a java.io.BufferedInputStream) at com.sun.jndi.ldap.Connection.run(Connection.java:834) at java.lang.Thread.run(Thread.java:662) "Thread-2384" daemon prio=10 tid=0x00002aaab8b67800 nid=0x3f46 runnable [0x000000004453d000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read1(BufferedInputStream.java:258) at java.io.BufferedInputStream.read(BufferedInputStream.java:317) - locked <0x00000007582bf988> (a java.io.BufferedInputStream) at com.sun.jndi.ldap.Connection.run(Connection.java:834) at java.lang.Thread.run(Thread.java:662) "Thread-2383" daemon prio=10 tid=0x00002aaab89b4000 nid=0x3f45 runnable [0x000000004443c000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read1(BufferedInputStream.java:258) at java.io.BufferedInputStream.read(BufferedInputStream.java:317) - locked <0x00000007582ed248> (a java.io.BufferedInputStream) at com.sun.jndi.ldap.Connection.run(Connection.java:834) at java.lang.Thread.run(Thread.java:662) "Thread-2382" daemon prio=10 tid=0x00002aaab908d800 nid=0x3f44 runnable [0x0000000045f85000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read1(BufferedInputStream.java:258) at java.io.BufferedInputStream.read(BufferedInputStream.java:317) - locked <0x000000075831ab08> (a java.io.BufferedInputStream) at com.sun.jndi.ldap.Connection.run(Connection.java:834) at java.lang.Thread.run(Thread.java:662) "Thread-2381" daemon prio=10 tid=0x00002aaab8e14800 nid=0x3f43 runnable [0x00000000406f3000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read1(BufferedInputStream.java:258) at java.io.BufferedInputStream.read(BufferedInputStream.java:317) - locked <0x00000007582ef420> (a java.io.BufferedInputStream) at com.sun.jndi.ldap.Connection.run(Connection.java:834) at java.lang.Thread.run(Thread.java:662) "pool-24-thread-1" prio=10 tid=0x0000000044edf800 nid=0x7de3 in Object.wait() [0x0000000046186000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.io.PipedInputStream.awaitSpace(PipedInputStream.java:257) at java.io.PipedInputStream.receive(PipedInputStream.java:215) - locked <0x000000077ed6f140> (a com.gc.iotools.stream.is.InputStreamFromOutputStream$MyPipedInputStream) at java.io.PipedOutputStream.write(PipedOutputStream.java:132) at java.util.zip.DeflaterOutputStream.deflate(DeflaterOutputStream.java:155) at java.util.zip.DeflaterOutputStream.write(DeflaterOutputStream.java:112) at java.util.zip.GZIPOutputStream.write(GZIPOutputStream.java:72) - locked <0x000000077ed898d8> (a java.util.zip.GZIPOutputStream) at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65) at java.io.BufferedOutputStream.write(BufferedOutputStream.java:109) - locked <0x000000077ee4a528> (a org.artifactory.io.checksum.ChecksumOutputStream) at org.artifactory.io.checksum.ChecksumOutputStream.write(ChecksumOutputStream.java:68) at com.ctc.wstx.io.UTF8Writer.write(UTF8Writer.java:143) at com.ctc.wstx.sw.BufferingXmlWriter.flushBuffer(BufferingXmlWriter.java:1366) at com.ctc.wstx.sw.BufferingXmlWriter.writeRaw(BufferingXmlWriter.java:239) at com.ctc.wstx.sw.BufferingXmlWriter.writeCharacters(BufferingXmlWriter.java:568) at com.ctc.wstx.sw.BaseStreamWriter.writeCharacters(BaseStreamWriter.java:462) at javanet.staxutils.IndentingXMLStreamWriter.writeCharacters(IndentingXMLStreamWriter.java:209) at com.thoughtworks.xstream.io.xml.StaxWriter.setValue(StaxWriter.java:152) at com.thoughtworks.xstream.io.WriterWrapper.setValue(WriterWrapper.java:45) at com.thoughtworks.xstream.converters.SingleValueConverterWrapper.marshal(SingleValueConverterWrapper.java:45) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:51) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:84) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshallField(AbstractReflectionConverter.java:225) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.writeField(AbstractReflectionConverter.java:204) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$2.<init>(AbstractReflectionConverter.java:167) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.doMarshal(AbstractReflectionConverter.java:116) at com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter.marshal(AbstractReflectionConverter.java:72) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:44) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller$1.convertAnother(AbstractReferenceMarshaller.java:88) at org.artifactory.addon.yum.model.common.PackageContainerConverter.marshal(Artifactory:41) at org.artifactory.addon.yum.model.primary.PrimaryConverter.marshal(Artifactory:37) at com.thoughtworks.xstream.core.AbstractReferenceMarshaller.convert(AbstractReferenceMarshaller.java:69) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:59) at com.thoughtworks.xstream.core.TreeMarshaller.convertAnother(TreeMarshaller.java:44) at com.thoughtworks.xstream.core.TreeMarshaller.start(TreeMarshaller.java:79) at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.marshal(AbstractTreeMarshallingStrategy.java:37) at com.thoughtworks.xstream.XStream.marshal(XStream.java:877) at com.thoughtworks.xstream.XStream.marshal(XStream.java:866) at com.thoughtworks.xstream.XStream.toXML(XStream.java:854) at org.artifactory.addon.yum.YumAddonImpl$1.produce(Artifactory:285) at com.gc.iotools.stream.is.InputStreamFromOutputStream$DataProducer.call(InputStreamFromOutputStream.java:108) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "art-exec-197" prio=10 tid=0x00000000457d8800 nid=0x7b54 in Object.wait() [0x0000000044138000] java.lang.Thread.State: BLOCKED (on object monitor) at java.lang.Object.wait(Native Method) at java.io.PipedInputStream.awaitSpace(PipedInputStream.java:257) at java.io.PipedInputStream.receive(PipedInputStream.java:215) - locked <0x000000077ed6f140> (a com.gc.iotools.stream.is.InputStreamFromOutputStream$MyPipedInputStream) at java.io.PipedOutputStream.write(PipedOutputStream.java:132) at java.util.zip.GZIPOutputStream.finish(GZIPOutputStream.java:95) at java.util.zip.DeflaterOutputStream.close(DeflaterOutputStream.java:140) at java.io.FilterOutputStream.close(FilterOutputStream.java:143) at org.artifactory.io.checksum.ChecksumOutputStream.close(ChecksumOutputStream.java:85) at org.apache.commons.io.IOUtils.closeQuietly(IOUtils.java:281) at org.apache.commons.io.IOUtils.closeQuietly(IOUtils.java:252) at org.artifactory.addon.yum.YumAddonImpl.a(Artifactory:299) at org.artifactory.addon.yum.YumAddonImpl.calculateYumMetadata(Artifactory:191) at org.artifactory.addon.yum.YumAddonImpl.executeAsyncRepositoryYumMetadataCalculation(Artifactory:154) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:318) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150) at org.artifactory.schedule.aop.AsyncAdvice$TraceableMethodInvocation.proceed(AsyncAdvice.java:320) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110) at org.artifactory.jcr.lock.aop.LockingAdvice.invoke(LockingAdvice.java:121) at org.artifactory.schedule.aop.AsyncAdvice.doInvoke(AsyncAdvice.java:198) at org.artifactory.schedule.aop.CompoundInvocation.proceed(CompoundInvocation.java:56) at org.artifactory.schedule.aop.AsyncAdvice.doInvoke(AsyncAdvice.java:184) at org.artifactory.schedule.aop.AsyncAdvice$1.call(AsyncAdvice.java:145) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at org.artifactory.schedule.ArtifactoryConcurrentExecutor$RunnableWrapper.run(ArtifactoryConcurrentExecutor.java:85) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-20" daemon prio=10 tid=0x0000000044e00000 nid=0x1c01 runnable [0x0000000044038000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000076e6ba798> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-19" daemon prio=10 tid=0x0000000045034800 nid=0x1c00 runnable [0x000000004322a000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x00000007b783b8a8> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-18" daemon prio=10 tid=0x00000000450d5000 nid=0x1bff runnable [0x000000004463e000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000076e6ba738> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-17" daemon prio=10 tid=0x000000004549b000 nid=0x1bfe runnable [0x000000004433b000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000077074df58> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-16" daemon prio=10 tid=0x0000000044d85800 nid=0x1bfd waiting on condition [0x0000000043f37000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007593ec1f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:104) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:32) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:662) "jackrabbit-pool-6" daemon prio=10 tid=0x00002aaab8cb0000 nid=0x4e6b waiting on condition [0x000000004473f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000075a92d020> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at org.apache.jackrabbit.core.RepositoryImpl$ThreadWrapper.run(RepositoryImpl.java:251) at java.lang.Thread.run(Thread.java:662) "MultiThreadedHttpConnectionManager cleanup" daemon prio=10 tid=0x00002aaac1a62800 nid=0x4ae2 in Object.wait() [0x000000004423a000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x000000075af79900> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118) - locked <0x000000075af79900> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134) at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122) "ajp-bio-8009-exec-15" daemon prio=10 tid=0x0000000044ea9000 nid=0x440b waiting on condition [0x0000000043e36000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007593ec1f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:104) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:32) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-14" daemon prio=10 tid=0x0000000045384800 nid=0x440a runnable [0x0000000043d35000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000076305b210> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-13" daemon prio=10 tid=0x00000000458fc800 nid=0x4409 runnable [0x0000000043c34000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000077041e3c8> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-12" daemon prio=10 tid=0x0000000045383800 nid=0x4408 runnable [0x0000000043b33000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000077074dc88> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-11" daemon prio=10 tid=0x0000000045184800 nid=0x4407 waiting on condition [0x0000000043a32000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007593ec1f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:104) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:32) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-10" daemon prio=10 tid=0x0000000045307800 nid=0x4406 runnable [0x0000000043931000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x00000007643ad428> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-9" daemon prio=10 tid=0x0000000044c81000 nid=0x4405 runnable [0x0000000043830000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000077b447c98> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-8" daemon prio=10 tid=0x000000004510c800 nid=0x43fc runnable [0x000000004372f000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x00000007aff520c8> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-7" daemon prio=10 tid=0x00000000453df800 nid=0x43fb waiting on condition [0x000000004362e000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007593ec1f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:104) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:32) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:662) "DynamicPooledExecutor" daemon prio=10 tid=0x000000004510c000 nid=0x43fa waiting on condition [0x000000004352d000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000075a01dc88> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-6" daemon prio=10 tid=0x0000000044b8d000 nid=0x43f3 waiting on condition [0x0000000043129000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007593ec1f0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:104) at org.apache.tomcat.util.threads.TaskQueue.take(TaskQueue.java:32) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-5" daemon prio=10 tid=0x0000000044ff7000 nid=0x43f1 runnable [0x0000000043028000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x00000007b112a1e8> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-4" daemon prio=10 tid=0x000000004532c800 nid=0x43f0 runnable [0x0000000042f27000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000077074dce8> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-3" daemon prio=10 tid=0x0000000045055800 nid=0x43ee runnable [0x0000000042e26000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x0000000773b4c018> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "Wicket-PageSavingThread" daemon prio=10 tid=0x000000004501d000 nid=0x43ec waiting on condition [0x0000000040d9e000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000075b6e7d98> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:424) at org.apache.wicket.pageStore.AsynchronousDataStore$PageSavingRunnable.run(AsynchronousDataStore.java:346) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-exec-2" daemon prio=10 tid=0x0000000044da6000 nid=0x43eb runnable [0x00000000417b7000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000076f995598> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "art-exec-1" prio=10 tid=0x00002aaac01bc000 nid=0x43dc waiting on condition [0x0000000042d25000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.artifactory.addon.event.AggregatingEventServiceImpl$EventSender.run(AggregatingEventServiceImpl.java:191) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at org.artifactory.schedule.ArtifactoryConcurrentExecutor$RunnableWrapper.run(ArtifactoryConcurrentExecutor.java:85) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "jackrabbit-pool-5" daemon prio=10 tid=0x00002aaac033e800 nid=0x43db waiting on condition [0x0000000042c24000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000075a92d020> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at org.apache.jackrabbit.core.RepositoryImpl$ThreadWrapper.run(RepositoryImpl.java:251) at java.lang.Thread.run(Thread.java:662) "scheduler_QuartzSchedulerThread" prio=10 tid=0x00002aaac032f000 nid=0x43da in Object.wait() [0x0000000042b23000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThread.java:433) - locked <0x000000075ae46010> (a java.lang.Object) "Timer-1" prio=10 tid=0x00002aaac00fd000 nid=0x43d8 in Object.wait() [0x0000000042a22000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.util.TimerThread.mainLoop(Timer.java:509) - locked <0x000000075ab42cd0> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462) "jackrabbit-pool-4" daemon prio=10 tid=0x00002aaac01ef000 nid=0x43d6 waiting on condition [0x0000000042921000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000075a92d020> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at org.apache.jackrabbit.core.RepositoryImpl$ThreadWrapper.run(RepositoryImpl.java:251) at java.lang.Thread.run(Thread.java:662) "jackrabbit-pool-3" daemon prio=10 tid=0x00002aaac0eca000 nid=0x43d4 waiting on condition [0x0000000042820000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000075a92d020> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at org.apache.jackrabbit.core.RepositoryImpl$ThreadWrapper.run(RepositoryImpl.java:251) at java.lang.Thread.run(Thread.java:662) "jackrabbit-pool-2" daemon prio=10 tid=0x00002aaac014c000 nid=0x43d3 waiting on condition [0x000000004271f000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000075a92d020> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at org.apache.jackrabbit.core.RepositoryImpl$ThreadWrapper.run(RepositoryImpl.java:251) at java.lang.Thread.run(Thread.java:662) "jackrabbit-pool-1" daemon prio=10 tid=0x00002aaac01f6800 nid=0x43d2 waiting on condition [0x000000004261e000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000000075a92d020> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:196) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:609) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:602) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) at org.apache.jackrabbit.core.RepositoryImpl$ThreadWrapper.run(RepositoryImpl.java:251) at java.lang.Thread.run(Thread.java:662) "ObservationManager" daemon prio=10 tid=0x0000000044e7d800 nid=0x43c6 in Object.wait() [0x00000000414a3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:485) at org.apache.commons.collections.buffer.BlockingBuffer.remove(BlockingBuffer.java:192) - locked <0x000000075a430d80> (a org.apache.commons.collections.buffer.BlockingBuffer) at org.apache.jackrabbit.core.observation.ObservationDispatcher.run(ObservationDispatcher.java:153) at java.lang.Thread.run(Thread.java:662) "Timer-0" daemon prio=10 tid=0x0000000044e6b000 nid=0x43c2 in Object.wait() [0x00000000418bf000] java.lang.Thread.State: TIMED_WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.util.TimerThread.mainLoop(Timer.java:509) - locked <0x000000075a3fc748> (a java.util.TaskQueue) at java.util.TimerThread.run(Timer.java:462) "ajp-bio-8009-exec-1" daemon prio=10 tid=0x00000000457ee000 nid=0x4262 runnable [0x0000000041bc2000] java.lang.Thread.State: RUNNABLE at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.read(SocketInputStream.java:129) at org.apache.coyote.ajp.AjpProcessor.read(AjpProcessor.java:309) at org.apache.coyote.ajp.AjpProcessor.readMessage(AjpProcessor.java:364) at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:128) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:565) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:307) - locked <0x000000076e6bc528> (a org.apache.tomcat.util.net.SocketWrapper) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-AsyncTimeout" daemon prio=10 tid=0x00002aaab88fe000 nid=0x4261 waiting on condition [0x0000000041ac1000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.tomcat.util.net.JIoEndpoint$AsyncTimeout.run(JIoEndpoint.java:148) at java.lang.Thread.run(Thread.java:662) "ajp-bio-8009-Acceptor-0" daemon prio=10 tid=0x00002aaab8643800 nid=0x4260 runnable [0x00000000419c0000] java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408) - locked <0x0000000759330a00> (a java.net.SocksSocketImpl) at java.net.ServerSocket.implAccept(ServerSocket.java:462) at java.net.ServerSocket.accept(ServerSocket.java:430) at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:60) at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:216) at java.lang.Thread.run(Thread.java:662) "ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon prio=10 tid=0x00002aaab8658800 nid=0x425f waiting on condition [0x00000000405f2000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1515) at java.lang.Thread.run(Thread.java:662) "logback-watchdog" daemon prio=10 tid=0x0000000044fe7000 nid=0x4251 waiting on condition [0x0000000040804000] java.lang.Thread.State: TIMED_WAITING (sleeping) at java.lang.Thread.sleep(Native Method) at org.artifactory.util.FileWatchDog.run(FileWatchDog.java:108) "Low Memory Detector" daemon prio=10 tid=0x0000000044a51000 nid=0x41c7 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread1" daemon prio=10 tid=0x0000000044a4e800 nid=0x41c6 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread0" daemon prio=10 tid=0x0000000044a49800 nid=0x41c5 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" daemon prio=10 tid=0x0000000044a47000 nid=0x41c4 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Surrogate Locker Thread (Concurrent GC)" daemon prio=10 tid=0x0000000044a45000 nid=0x41c3 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Finalizer" daemon prio=10 tid=0x0000000044a23000 nid=0x41c2 in Object.wait() [0x000000004231b000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118) - locked <0x0000000759339c90> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) "Reference Handler" daemon prio=10 tid=0x0000000044a21000 nid=0x41c1 in Object.wait() [0x000000004221a000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:485) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) - locked <0x00000007593308c8> (a java.lang.ref.Reference$Lock) "main" prio=10 tid=0x00000000448f6800 nid=0x41b9 runnable [0x000000004109f000] java.lang.Thread.State: RUNNABLE at java.net.PlainSocketImpl.socketAccept(Native Method) at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408) - locked <0x00000007594d5268> (a java.net.SocksSocketImpl) at java.net.ServerSocket.implAccept(ServerSocket.java:462) at java.net.ServerSocket.accept(ServerSocket.java:430) at org.apache.catalina.core.StandardServer.await(StandardServer.java:452) at org.apache.catalina.startup.Catalina.await(Catalina.java:757) at org.apache.catalina.startup.Catalina.start(Catalina.java:703) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:322) at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:450) "VM Thread" prio=10 tid=0x0000000044a1a800 nid=0x41c0 runnable "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x0000000044905000 nid=0x41ba runnable "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x0000000044906800 nid=0x41bb runnable "Gang worker#2 (Parallel GC Threads)" prio=10 tid=0x0000000044908800 nid=0x41bc runnable "Gang worker#3 (Parallel GC Threads)" prio=10 tid=0x000000004490a800 nid=0x41bd runnable "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x0000000044999800 nid=0x41be runnable "VM Periodic Task Thread" prio=10 tid=0x0000000044a5c000 nid=0x41c8 waiting on condition JNI global references: 1609 Heap par new generation total 76672K, used 16484K [0x0000000754000000, 0x0000000759330000, 0x0000000759330000) eden space 68160K, 17% used [0x0000000754000000, 0x0000000754bfa1f0, 0x0000000758290000) from space 8512K, 49% used [0x0000000758290000, 0x00000007586aee38, 0x0000000758ae0000) to space 8512K, 0% used [0x0000000758ae0000, 0x0000000758ae0000, 0x0000000759330000) concurrent mark-sweep generation total 2536256K, used 2106541K [0x0000000759330000, 0x00000007f4000000, 0x00000007f4000000) concurrent-mark-sweep perm gen total 196608K, used 96419K [0x00000007f4000000, 0x0000000800000000, 0x0000000800000000) ---- [3] System Information: System Properties: Wicket_HeaderRenderStrategy | org.apache.wicket.markup.renderStrategy.ParentFirstHeaderRenderStrategy artifactory.home | /var/lib/artifactory artifactory.jcr.configDir | repo/filesystem-mysql catalina.base | /opt/artifactory/tomcat catalina.home | /opt/artifactory/tomcat catalina.useNaming | true common.loader | ${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar derby.language.logStatementText | false derby.module.mgmt.jmx | org.apache.derby.impl.services.jmxnone.NoManagementService derby.storage.pageCacheSize | 500 derby.stream.error.file | /var/lib/artifactory/logs/derby.log derby.stream.error.logSeverityLevel | 0 file.encoding | UTF-8 file.encoding.pkg | sun.io file.separator | / java.awt.graphicsenv | sun.awt.X11GraphicsEnvironment java.awt.printerjob | sun.print.PSPrinterJob java.class.path | /opt/artifactory/tomcat/bin/bootstrap.jar:/opt/artifactory/tomcat/bin/tomcat-juli.jar java.class.version | 50.0 java.endorsed.dirs | /opt/artifactory/tomcat/endorsed java.ext.dirs | /usr/java/jdk1.6.0_33/jre/lib/ext:/usr/java/packages/lib/ext java.home | /usr/java/jdk1.6.0_33/jre java.io.tmpdir | /opt/artifactory/tomcat/temp java.library.path | /usr/java/jdk1.6.0_33/jre/lib/amd64/server:/usr/java/jdk1.6.0_33/jre/lib/amd64:/usr/java/jdk1.6.0_33/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib java.naming.factory.initial | org.apache.naming.java.javaURLContextFactory java.naming.factory.url.pkgs | org.apache.naming java.runtime.name | Java(TM) SE Runtime Environment java.runtime.version | 1.6.0_33-b04 java.specification.name | Java Platform API Specification java.specification.vendor | Sun Microsystems Inc. java.specification.version | 1.6 java.util.logging.config.file | /opt/artifactory/tomcat/conf/logging.properties java.util.logging.manager | org.apache.juli.ClassLoaderLogManager java.vendor | Sun Microsystems Inc. java.vendor.url | http://java.sun.com/ java.vendor.url.bug | http://java.sun.com/cgi-bin/bugreport.cgi java.version | 1.6.0_33 java.vm.info | mixed mode java.vm.name | Java HotSpot(TM) 64-Bit Server VM java.vm.specification.name | Java Virtual Machine Specification java.vm.specification.vendor | Sun Microsystems Inc. java.vm.specification.version | 1.0 java.vm.vendor | Sun Microsystems Inc. java.vm.version | 20.8-b03 line.separator | \n org.apache.jackrabbit.core.util.RepositoryLock:/var/lib/artifactory/data| org.apache.jackrabbit.core.util.RepositoryLock:/var/lib/artifactory/data org.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER| true os.arch | amd64 os.name | Linux os.version | 2.6.18-308.1.1.el5.centos.plus package.access | sun.,org.apache.catalina.,org.apache.coyote.,org.apache.tomcat.,org.apache.jasper. package.definition | sun.,java.,org.apache.catalina.,org.apache.coyote.,org.apache.tomcat.,org.apache.jasper. path.separator | : server.loader | shared.loader | sun.arch.data.model | 64 sun.boot.class.path | /usr/java/jdk1.6.0_33/jre/lib/resources.jar:/usr/java/jdk1.6.0_33/jre/lib/rt.jar:/usr/java/jdk1.6.0_33/jre/lib/sunrsasign.jar:/usr/java/jdk1.6.0_33/jre/lib/jsse.jar:/usr/java/jdk1.6.0_33/jre/lib/jce.jar:/usr/java/jdk1.6.0_33/jre/lib/charsets.jar:/usr/java/jdk1.6.0_33/jre/lib/modules/jdk.boot.jar:/usr/java/jdk1.6.0_33/jre/classes sun.boot.library.path | /usr/java/jdk1.6.0_33/jre/lib/amd64 sun.cpu.endian | little sun.cpu.isalist | sun.io.unicode.encoding | UnicodeLittle sun.java.command | org.apache.catalina.startup.Bootstrap start sun.java.launcher | SUN_STANDARD sun.jnu.encoding | UTF-8 sun.management.compiler | HotSpot 64-Bit Tiered Compilers sun.os.patch.level | unknown tomcat.util.buf.StringCache.byte.enabled | true tomcat.util.scan.DefaultJarScanner.jarsToSkip | bootstrap.jar,commons-daemon.jar,tomcat-juli.jar,annotations-api.jar,el-api.jar,jsp-api.jar,servlet-api.jar,catalina.jar,catalina-ant.jar,catalina-ha.jar,catalina-tribes.jar,jasper.jar,jasper-el.jar,ecj-*.jar,tomcat-api.jar,tomcat-util.jar,tomcat-coyote.jar,tomcat-dbcp.jar,tomcat-i18n-en.jar,tomcat-i18n-es.jar,tomcat-i18n-fr.jar,tomcat-i18n-ja.jar,tomcat-juli-adapters.jar,catalina-jmx-remote.jar,catalina-ws.jar,tomcat-jdbc.jar,commons-beanutils*.jar,commons-codec*.jar,commons-collections*.jar,commons-dbcp*.jar,commons-digester*.jar,commons-fileupload*.jar,commons-httpclient*.jar,commons-io*.jar,commons-lang*.jar,commons-logging*.jar,commons-math*.jar,commons-pool*.jar,jstl.jar,geronimo-spec-jaxrpc*.jar,wsdl4j*.jar,ant.jar,ant-junit*.jar,aspectj*.jar,jmx.jar,h2*.jar,hibernate*.jar,httpclient*.jar,jmx-tools.jar,jta*.jar,log4j*.jar,mail*.jar,slf4j*.jar,xercesImpl.jar,xmlParserAPIs.jar,xml-apis.jar,dnsns.jar,ldapsec.jar,localedata.jar,sunjce_provider.jar,sunmscapi.jar,sunpkcs11.jar,jhall.jar,tools.jar,sunec.jar,zipfs.jar,apple_provider.jar,AppleScriptEngine.jar,CoreAudio.jar,dns_sd.jar,j3daudio.jar,j3dcore.jar,j3dutils.jar,jai_core.jar,jai_codec.jar,mlibwrapper_jai.jar,MRJToolkit.jar,vecmath.jar,junit.jar,junit-*.jar,ant-launcher.jar user.country | US user.dir | / user.home | /home/builder user.language | en user.name | builder user.timezone | America/Los_Angeles General JVM Info: Available Processors | 4 Heap Memory Usage-Committed | 2675638272 Heap Memory Usage-Init | 2684354560 Heap Memory Usage-Max | 2675638272 Heap Memory Usage-Used | 1001131216 Non-Heap Memory Usage-Committed | 220921856 Non-Heap Memory Usage-Init | 203882496 Non-Heap Memory Usage-Max | 251658240 Non-Heap Memory Usage-Used | 119283448 JVM Arguments: -Djava.util.logging.config.file=/opt/artifactory/tomcat/conf/logging.properties -Xms2560m -Xmx2560m -XX:PermSize=192m -XX:MaxPermSize=192m -XX:+UseParNewGC -XX:+UseTLAB -XX:+UseConcMarkSweepGC -Dartifactory.home=/var/lib/artifactory -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Dorg.apache.jasper.runtime.BodyContentImpl.LIMIT_BUFFER=true -Djava.endorsed.dirs=/opt/artifactory/tomcat/endorsed -Dcatalina.base=/opt/artifactory/tomcat -Dcatalina.home=/opt/artifactory/tomcat -Djava.io.tmpdir=/opt/artifactory/tomcat/temp ---- |