[Jetty-support] Acceptor thread taking nearly 100% CPU

[Chris S. <chr...@gm...>]

(Apologies if this is a double post: think I might have sent this to the
wrong mailing list first time)

I'm seeing a situation on our server where Jetty's acceptor thread is using
nearly 100% CPU.  As far as I can tell it is just spinning so it is
relatively harmless as other threads get time, but makes monitoring actual
server loading a bit hard!

Looking at the mailing lists and JIRA there seem to have been a few similar
issues reported in the past but they look to be resolved.  I can see nothing
in the change logs between the release we're using and the latest that says
to me any relevant fixes and I can't /easily/ update Jetty just to see if it
works so I thought I'd ask.  We're running:

Jetty 6.1.11
JDK 1.6.0_11
CentOS 5.2 (2.6.18-92.1.18.el5)

We had this on the past on a four core box but it solved itself after a day
or so. Since we have switched to an 8 core box the problem is permanent.
Happens some time after startup.  Jstack for the offending thread:

"btpool2-3 - Acceptor0 SelectChannelConnector@0.0.0.0:80" prio=10
tid=0x00002aacb79c0000 nid=0x2a8d runnable [0x000000004252c000..0x
000000004252ca10]
   java.lang.Thread.State: RUNNABLE
        at java.lang.Object.notify(Native Method)
        at
org.mortbay.thread.QueuedThreadPool$PoolThread.dispatch(QueuedThreadPool.java:583)
        - locked <0x00002aacafa97670> (a
org.mortbay.thread.QueuedThreadPool$PoolThread)
        at
org.mortbay.thread.QueuedThreadPool.dispatch(QueuedThreadPool.java:114)
        at
org.mortbay.jetty.nio.SelectChannelConnector$1.dispatch(SelectChannelConnector.java:86)
        at
org.mortbay.io.nio.SelectChannelEndPoint.dispatch(SelectChannelEndPoint.java:67)
        at
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:476)
        at
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:167)
        at
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
        at
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:707)
        at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:488)

It's a plan HTTP acceptor (not SSL) but it is using Continuations.

Any advice or suggestions appreciated.

Thanks,

Chris

P.S. First post to the mailing list after years of using Jetty.  Compact,
easy to use, powerful, used it for all sorts of things. Great product! :-)