[Webmacro-devel] Deadlock

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Hi, I have been given the task of finding out the cause of some
threading problems in one of or applications. The application is you run
of the mill J2EE app. and uses webmacro version 1.1 as its templating
engine. We are running JBoss-3.2.3 with integrated jetty on windows 2000
server with jdk 1.4.1_02

Recently the server decided to hang and only recovered on a reboot,
after looking through the log files I noticed Jetty was complaining
about a lack of threads. After profiling the server at one point I
managed to bring the server down (by simulating traffic). I dumped the
threads to console (ctrl-break in windows) and it indicated there was
one deadlock the output is below:

Found one Java-level deadlock:
=============================
"PoolThread-14":
  waiting to lock monitor 0x8f6dfc (object 0x3c45720, a
java.lang.Object),
  which is held by "PoolThread-11"
"PoolThread-11":
  waiting to lock monitor 0x8f6c7c (object 0x3c45758, a
java.lang.Object),
  which is held by "PoolThread-14"

Java stack information for the threads listed above:
===================================================
"PoolThread-14":
        at
org.webmacro.resource.SMapCacheManager.get(SMapCacheManager.java:238)

        - waiting to lock <03C45720> (a java.lang.Object)
        at
org.webmacro.resource.CachingProvider.get(CachingProvider.java:101)
        at org.webmacro.Broker.get(Broker.java:658)
        at
org.webmacro.directive.IncludeDirective.getTemplate(IncludeDirective.
java:459)
        at
org.webmacro.directive.IncludeDirective.build(IncludeDirective.java:2
95)
        at
org.webmacro.directive.DirectiveBuilder.build(DirectiveBuilder.java:2
19)
        at
org.webmacro.engine.BlockBuilder.build(BlockBuilder.java:126)
        at org.webmacro.engine.WMTemplate.parse(WMTemplate.java:168)
        at
org.webmacro.resource.BrokerTemplateProviderHelper.load(BrokerTemplat
eProviderHelper.java:102)
        at
org.webmacro.resource.TemplateProvider.load(TemplateProvider.java:175
)
        at
org.webmacro.resource.CachingProvider.load(CachingProvider.java:111)
        at
org.webmacro.resource.SMapCacheManager.get(SMapCacheManager.java:244)

        - locked <03C45758> (a java.lang.Object)
        at
org.webmacro.resource.CachingProvider.get(CachingProvider.java:101)
        at org.webmacro.Broker.get(Broker.java:658)
        at
org.webmacro.directive.IncludeDirective.getTemplate(IncludeDirective.
java:459)
        at
org.webmacro.directive.IncludeDirective.build(IncludeDirective.java:2
95)
        at
org.webmacro.directive.DirectiveBuilder.build(DirectiveBuilder.java:2
19)
        at
org.webmacro.engine.BlockBuilder.build(BlockBuilder.java:126)
        at org.webmacro.engine.WMTemplate.parse(WMTemplate.java:168)
        at
org.webmacro.resource.BrokerTemplateProviderHelper.load(BrokerTemplat
eProviderHelper.java:102)
        at
org.webmacro.resource.TemplateProvider.load(TemplateProvider.java:175
)
        at
org.webmacro.resource.CachingProvider.load(CachingProvider.java:111)
        at
org.webmacro.resource.SMapCacheManager.get(SMapCacheManager.java:244)

        - locked <03C45878> (a java.lang.Object)
        at
org.webmacro.resource.CachingProvider.get(CachingProvider.java:101)
        at org.webmacro.Broker.get(Broker.java:658)
        at
org.webmacro.directive.IncludeDirective.getTemplate(IncludeDirective.
java:459)
        at
org.webmacro.directive.IncludeDirective.getThingToInclude(IncludeDire
ctive.java:433)
        at
org.webmacro.directive.IncludeDirective.write(IncludeDirective.java:3
54)
        at org.webmacro.engine.Block.write(Block.java:141)
        at
org.webmacro.directive.IfDirective.write(IfDirective.java:192)
        at org.webmacro.engine.Block.write(Block.java:194)
        at org.webmacro.engine.WMTemplate.write(WMTemplate.java:262)
        at
org.webmacro.directive.IncludeDirective.write(IncludeDirective.java:3
63)
        at org.webmacro.engine.Block.write(Block.java:141)
        at
org.webmacro.directive.IfDirective.write(IfDirective.java:192)
        at org.webmacro.engine.Block.write(Block.java:194)
        at org.webmacro.engine.WMTemplate.write(WMTemplate.java:262)
        at org.webmacro.servlet.WMServlet.execute(WMServlet.java:524)
        at
org.webmacro.servlet.WMServlet.doRequest(WMServlet.java:262)
        at org.webmacro.servlet.WMServlet.doGet(WMServlet.java:195)
        at
javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
        at
javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:360
)
        at
org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApp
licationHandler.java:342)
        at
com.msxi.fcg.servlet.filter.LoggedInFilter.doFilter(LoggedInFilter.ja
va:53)
        at
org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApp
licationHandler.java:334)
        at
com.msxi.util.filter.CompressionFilter.doFilter(CompressionFilter.jav
a:78)
        at
org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApp
licationHandler.java:334)
        at
org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicati
onHandler.java:286)
        at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:5
58)
        at org.mortbay.http.HttpContext.handle(HttpContext.java:1714)
        at
org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplication
Context.java:505)
        at org.mortbay.http.HttpContext.handle(HttpContext.java:1664)
        at org.mortbay.http.HttpServer.service(HttpServer.java:879)
        at org.jboss.jetty.Jetty.service(Jetty.java:460)
        at
org.mortbay.http.HttpConnection.service(HttpConnection.java:770)
        at
org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:932)
        at
org.mortbay.http.HttpConnection.handle(HttpConnection.java:787)
        at
org.mortbay.http.SocketListener.handleConnection(SocketListener.java:
201)
        at
org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:289)
        at
org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:454)
"PoolThread-11":
        at
org.webmacro.resource.SMapCacheManager.get(SMapCacheManager.java:238)

        - waiting to lock <03C45758> (a java.lang.Object)
        at
org.webmacro.resource.CachingProvider.get(CachingProvider.java:101)
        at org.webmacro.Broker.get(Broker.java:658)
        at
org.webmacro.directive.IncludeDirective.getTemplate(IncludeDirective.
java:459)
        at
org.webmacro.directive.IncludeDirective.build(IncludeDirective.java:2
95)
        at
org.webmacro.directive.DirectiveBuilder.build(DirectiveBuilder.java:2
19)
        at
org.webmacro.engine.BlockBuilder.build(BlockBuilder.java:126)
        at org.webmacro.engine.WMTemplate.parse(WMTemplate.java:168)
        at
org.webmacro.resource.BrokerTemplateProviderHelper.load(BrokerTemplat
eProviderHelper.java:102)
        at
org.webmacro.resource.TemplateProvider.load(TemplateProvider.java:175
)
        at
org.webmacro.resource.CachingProvider.load(CachingProvider.java:111)
        at
org.webmacro.resource.SMapCacheManager.get(SMapCacheManager.java:244)

        - locked <03C45720> (a java.lang.Object)
        at
org.webmacro.resource.CachingProvider.get(CachingProvider.java:101)
        at org.webmacro.Broker.get(Broker.java:658)
        at
org.webmacro.directive.IncludeDirective.getTemplate(IncludeDirective.
java:459)
        at
org.webmacro.directive.IncludeDirective.getThingToInclude(IncludeDire
ctive.java:433)
        at
org.webmacro.directive.IncludeDirective.write(IncludeDirective.java:3
54)
        at org.webmacro.engine.Block.write(Block.java:141)
        at
org.webmacro.directive.IfDirective.write(IfDirective.java:192)
        at org.webmacro.engine.Block.write(Block.java:194)
        at org.webmacro.engine.WMTemplate.write(WMTemplate.java:262)
        at
org.webmacro.directive.IncludeDirective.write(IncludeDirective.java:3
63)
        at org.webmacro.engine.Block.write(Block.java:141)
        at
org.webmacro.directive.IfDirective.write(IfDirective.java:192)
        at org.webmacro.engine.Block.write(Block.java:194)
        at org.webmacro.engine.WMTemplate.write(WMTemplate.java:262)
        at org.webmacro.servlet.WMServlet.execute(WMServlet.java:524)
        at
org.webmacro.servlet.WMServlet.doRequest(WMServlet.java:262)
        at org.webmacro.servlet.WMServlet.doPost(WMServlet.java:212)
        at
javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
        at
javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:360
)
        at
org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicati
onHandler.java:294)
        at
org.mortbay.jetty.servlet.Dispatcher.dispatch(Dispatcher.java:206)
        at
org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:133)
        at
com.msxi.fcg.servlet.helper.FCGHelper.redirectTo(FCGHelper.java:148)
        at
com.msxi.fcg.servlet.helper.DocumentServiceHelperImpl.putResults(Docu
mentServiceHelperImpl.java:380)
        at
com.msxi.fcg.servlet.helper.DocumentServiceHelperImpl.doDocumentSearc
h(DocumentServiceHelperImpl.java:132)
        at
com.msxi.fcg.servlet.DocumentServlet.doSearch(DocumentServlet.java:16
6)
        at
com.msxi.fcg.servlet.DocumentServlet.handleRequest(DocumentServlet.ja
va:122)
        at com.msxi.fcg.servlet.FCGServlet.handle(FCGServlet.java:164)
        at
org.webmacro.servlet.WMServlet.doRequest(WMServlet.java:255)
        at org.webmacro.servlet.WMServlet.doPost(WMServlet.java:212)
        at
javax.servlet.http.HttpServlet.service(HttpServlet.java:760)
        at
javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:360
)
        at
org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApp
licationHandler.java:342)
        at
com.msxi.fcg.servlet.filter.LoggedInFilter.doFilter(LoggedInFilter.ja
va:53)
        at
org.mortbay.jetty.servlet.WebApplicationHandler$Chain.doFilter(WebApp
licationHandler.java:334)
        at
org.mortbay.jetty.servlet.WebApplicationHandler.dispatch(WebApplicati
onHandler.java:286)
        at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:5
58)
        at org.mortbay.http.HttpContext.handle(HttpContext.java:1714)
        at
org.mortbay.jetty.servlet.WebApplicationContext.handle(WebApplication
Context.java:505)
        at org.mortbay.http.HttpContext.handle(HttpContext.java:1664)
        at org.mortbay.http.HttpServer.service(HttpServer.java:879)
        at org.jboss.jetty.Jetty.service(Jetty.java:460)
        at
org.mortbay.http.HttpConnection.service(HttpConnection.java:770)
        at
org.mortbay.http.HttpConnection.handleNext(HttpConnection.java:932)
        at
org.mortbay.http.HttpConnection.handle(HttpConnection.java:787)
        at
org.mortbay.http.SocketListener.handleConnection(SocketListener.java:
201)
        at
org.mortbay.util.ThreadedServer.handle(ThreadedServer.java:289)
        at
org.mortbay.util.ThreadPool$PoolThread.run(ThreadPool.java:454)

Found 1 deadlock.

Now I doubt this is the root cause of my problems (though it may well
be:( However it is quite unnerving that this should happen. Do you think
it because of some code we may have written or could it be a bug in
webmacro itself Or Jetty? I would guess at jetty since I have noticed
that it is producing quite a few blocked threads largely on redirects
(which is trying to occur above). Unfortunately I do not have time to
write a simple test case, but I would appreciate any feedback you can
give me. 

keep up the good work... and thanks in advance.

Laurence
This Message has been Checked at MSXI for all known Viruses.
You open this at your own risk. Please make sure all replies are 
also virus free.
Also we do not accept or send Attachments of the type .exe, .vbs,
scr, or .bat due to the virus risk they can contain. These types of
attachments will be stripped from the message.

MSXI