Menu
▾
▴
#1259 Deadlock between main thread and JS executor
Hello again dear HtmlUnit maintainers,
I ran into the following problem while running the integration test suite for my application. The problem was intermittent and very difficult to track down. The symptoms of the problem were: elements missing from the page, the entire dom missing and the browser simply having the wrong page loaded in the window. So I cranked up the logging and was able to find the culprit. When the click is being process on the "main" thread after the actual response is downloaded it's added to the loadQueue_ in the WebClient to be processed later when the JavaScriptEngine.processPostponedActions method is invoked. This then invokes the JavaScriptEngine.doProcessPostponedActions and then the page is loaded into the target window in the WebClient.loadDownloadedResponses method. Now the JavaScriptEngine.doProcessPostponedActions is also invoked after scheduled java script jobs are executed. So what was happening in my application is that the click would add a load job on the "main" thread but that job would be picked off the queue by the "JS executor ..." thread. The WebClient.loadDownloadedResponse method will return if the loadQueue_ is empty so the "main" thread was returning before the page was actually loaded into the target window. The attached file contains a unit test that exposes the issue and the fix to the WebClient.loadDownloadedResponses implementation. I hope that I correctly understood the desired behavior for that method and that the fix I've provided is appropriate.
Thank you.
Which version/build do you use?
I've used latest SVN version.
strange, i did some fixes for the deadlock by change 7645 and 7609.
Can you please verify that you are really running the current source and attach the whole thread dump.
I've checked out today code using information from "code" section: svn checkout http://svn.code.sf.net/p/htmlunit/code/trunk htmlunit-code.
After checkout I checked if changes mentioned in summary of latest build on HtmlUnit Cruise Control are visible and they were, so I believe I'm running current source.
I will provide full thread dump on Monday. Unfortunately I didn't found the minimal test case, but during debug it seemed that deadlock appeared after processing onload event on page from web response. Moreover it only appeared for FireFox emulators and as I mentioned earlier for our case problem disappeared when I removed "synchronized" keyword from JavaScripEngine.doProcessPostponedActions method signature.
It is really important for our tests to resolve this issue, so if you need any additional info I will do my best. Any help would be greatly appreciated, because as I said I find it difficult to assess the impact of this code change.
Adam,
guess you are right. After some more thinking i'm sure there are some possible scenarios for a deadlock. But i really need the thread dump for a deeper analysis.
If there is a (security) problem with sending the detailed dump, you can use private mail.
A small change is already checked in but I'm not sure that this fix solves all problems.
will have a look at it over the weekend
Here's the stack trace:
JavaScriptEngine.doProcessPostponedActions(boolean) line: 688
JavaScriptEngine.access$400(JavaScriptEngine, boolean) line: 89
JavaScriptEngine$4(JavaScriptEngine$HtmlUnitContextAction).run(Context) line: 659
Context.call(ContextFactory, ContextAction) line: 559
HtmlUnitContextFactory(ContextFactory).call(ContextAction) line: 525
JavaScriptEngine.callFunction(HtmlPage, Function, Scriptable, Scriptable, Object[]) line: 594
JavaScriptEngine.callFunction(HtmlPage, Function, Scriptable, Object[], DomNode) line: 569
HtmlPage.executeJavaScriptFunctionIfPossible(Function, Scriptable, Object[], DomNode) line: 996
EventListenersContainer.executeEventHandler(Event, Object[]) line: 208
EventListenersContainer.executeListeners(Event, Object[], Object[]) line: 292
Window.executeEvent(Event, EventListenersContainer) line: 1967
HTMLImageElement(Node).executeEvent(Event) line: 725
HtmlImage.doOnLoad() line: 182
HtmlImage.setAttributeNS(String, String, String) line: 114
HtmlImage(DomElement).setAttribute(String, String) line: 292
HTMLImageElement.setSrc(String) line: 95
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25
Method.invoke(Object, Object...) line: 597
MemberBox.invoke(Object, Object[]) line: 137
ScriptableObject$GetterSlot.setValue(Object, Scriptable, Scriptable) line: 287
HTMLImageElement(ScriptableObject).putImpl(String, int, Scriptable, Object) line: 2659
HTMLImageElement(ScriptableObject).put(String, Scriptable, Object) line: 509
ScriptableObject.putProperty(Scriptable, String, Object) line: 2364
ScriptRuntime.setObjectProp(Scriptable, String, Object, Context) line: 1601
ScriptRuntime.setObjectProp(Object, String, Object, Context) line: 1595
Interpreter.interpretLoop(Context, Interpreter$CallFrame, Object) line: 1248
Interpreter.interpret(InterpretedFunction, Context, Scriptable, Scriptable, Object[]) line: 815
InterpretedFunction.call(Context, Scriptable, Scriptable, Object[]) line: 109
HtmlUnitContextFactory(ContextFactory).doTopCall(Callable, Context, Scriptable, Scriptable, Object[]) line: 415
HtmlUnitContextFactory.doTopCall(Callable, Context, Scriptable, Scriptable, Object[]) line: 274
ScriptRuntime.doTopCall(Callable, Context, Scriptable, Scriptable, Object[]) line: 3132
InterpretedFunction.call(Context, Scriptable, Scriptable, Object[]) line: 107
JavaScriptEngine$4.doRun(Context) line: 587
JavaScriptEngine$4(JavaScriptEngine$HtmlUnitContextAction).run(Context) line: 651
Context.call(ContextFactory, ContextAction) line: 559
HtmlUnitContextFactory(ContextFactory).call(ContextAction) line: 525
JavaScriptEngine.callFunction(HtmlPage, Function, Scriptable, Scriptable, Object[]) line: 594
JavaScriptEngine.callFunction(HtmlPage, Function, Scriptable, Object[], DomNode) line: 569
HtmlPage.executeJavaScriptFunctionIfPossible(Function, Scriptable, Object[], DomNode) line: 996
EventListenersContainer.executeEventListeners(boolean, Event, Object[]) line: 175
EventListenersContainer.executeBubblingListeners(Event, Object[], Object[]) line: 234
Node.fireEvent(SimpleScriptable, Event) line: 813
HTMLHtmlElement(Node).fireEvent(Event) line: 737
HtmlElement$1.run(Context) line: 853
Context.call(ContextFactory, ContextAction) line: 559
HtmlUnitContextFactory(ContextFactory).call(ContextAction) line: 525
HtmlHtml(HtmlElement).fireEvent(Event) line: 858
HtmlPage.executeEventHandlersIfNeeded(String) line: 1259
HtmlPage.cleanUp() line: 278
WebClient.loadWebResponseInto(WebResponse, WebWindow) line: 429
WebClient.getPage(WebWindow, WebRequest) line: 308
WebClient.getPage(URL) line: 373
WebClient.getPage(String) line: 358
JavaScriptEngine$3(JavaScriptEngine$HtmlUnitContextAction).run(Context) line: 648
Context.call(ContextFactory, ContextAction) line: 559
HtmlUnitContextFactory(ContextFactory).call(ContextAction) line: 525
JavaScriptEngine.execute(HtmlPage, Script) line: 548
HtmlPage.loadExternalJavaScriptFile(String, String) line: 1073
HtmlScript.executeScriptIfNeeded() line: 399
HtmlScript$3.execute() line: 266
JavaScriptEngine.doProcessPostponedActions(boolean) line: 712
JavaScriptEngine.access$400(JavaScriptEngine, boolean) line: 89
JavaScriptEngine$4(JavaScriptEngine$HtmlUnitContextAction).run(Context) line: 659
Context.call(ContextFactory, ContextAction) line: 559
HtmlUnitContextFactory(ContextFactory).call(ContextAction) line: 525
JavaScriptEngine.callFunction(HtmlPage, Function, Scriptable, Scriptable, Object[]) line: 594
JavaScriptEngine.callFunction(HtmlPage, Function, Scriptable, Object[], DomNode) line: 569
HtmlPage.executeJavaScriptFunctionIfPossible(Function, Scriptable, Object[], DomNode) line: 996
JavaScriptFunctionJob.runJavaScript(HtmlPage) line: 53
JavaScriptFunctionJob(JavaScriptExecutionJob).run() line: 101
JavaScriptJobManagerImpl.runSingleJob(JavaScriptJob) line: 328
DefaultJavaScriptExecutor.run() line: 161
Thread.run() line: 680
Hi Adam,
any chance to get the complete thread dump (not the stack trace)?
If yes, please attach a file instead of pasting it.
Threaddump from deadlock situation.
Thanks for the dump.
Can you please the latest snapshot.
I guess, the deadlock situation from the dump is fixed since the weekend.
If not, please create a dump with the latest code from SVN.
Unfortunately deadlock still occurs. I think the "synchronized" keyword is misplaced - it should be added to "public void processPostponedActions()" not to "private void doProcessPostponedActions()". See next threaddump.
Last edit: Adam Afeltowicz 2012-12-04
The sync is now completely gone.
Please try and report back.
Seems ok now.
Thanks, will have a look at it in the evening.
Fixed.
Fixed.
Could someone please comment on what release the fixes mentioned here went out with. By looking at the Change history (http://htmlunit.sourceforge.net/changes-report.html#a2.11) you might think v2.11 Nov 6, 2012.
But the 'Fixed' comments here are dated 2012-12-26, after the 2.11 release date. Also, discussions about the issue continued after the v2.11 release date. And the "Milestone" is set to "Latest SVN" so that doesn't help either. I look at WebClient commits around December 2012 and didn't see any related to this issue either so I'm wondering what happened.
thanks.
Hi guys,
I'm playing around with 2.19 and this is still happenning.
I was about to open a new ticket but maybe this one contains useful information for fixing this? I'm attaching a deadlock report (I've got plenty more in case they help).
Found one Java-level deadlock:
"JS executor for com.gargoylesoftware.htmlunit.WebClient@3970bb27":
waiting to lock monitor 0x00007fbb5dc5ff18 (object 0x0000000670d3fe10, a com.gargoylesoftware.htmlunit.html.HtmlPage),
which is held by “main”
“main“:
waiting to lock monitor 0x00007fbbac777b28 (object 0x00000006749e6fd0, a com.gargoylesoftware.htmlunit.html.HtmlPage),
which is held by "JS executor for com.gargoylesoftware.htmlunit.WebClient@3970bb27"
Java stack information for the threads listed above:
"JS executor for com.gargoylesoftware.htmlunit.WebClient@3970bb27":
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:935)
“main”:
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine$HtmlUnitContextAction.run(JavaScriptEngine.java:935)
- waiting to lock <0x00000006749e6fd0> (a com.gargoylesoftware.htmlunit.html.HtmlPage)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:628)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:513)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:885)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.callFunction(JavaScriptEngine.java:857)
at com.gargoylesoftware.htmlunit.InteractivePage.executeJavaScriptFunctionIfPossible(InteractivePage.java:216)
at com.gargoylesoftware.htmlunit.javascript.host.event.EventListenersContainer.executeEventListeners(EventListenersContainer.java:260)
at com.gargoylesoftware.htmlunit.javascript.host.event.EventListenersContainer.executeBubblingListeners(EventListenersContainer.java:324)
at com.gargoylesoftware.htmlunit.javascript.host.event.EventTarget.fireEvent(EventTarget.java:221)
at com.gargoylesoftware.htmlunit.html.DomElement$2.run(DomElement.java:1160)
at net.sourceforge.htmlunit.corejs.javascript.Context.call(Context.java:628)
at net.sourceforge.htmlunit.corejs.javascript.ContextFactory.call(ContextFactory.java:513)
at com.gargoylesoftware.htmlunit.html.DomElement.fireEvent(DomElement.java:1165)
at com.gargoylesoftware.htmlunit.html.HtmlPage.executeEventHandlersIfNeeded(HtmlPage.java:1237)
at com.gargoylesoftware.htmlunit.html.HtmlPage.cleanUp(HtmlPage.java:332)
at com.gargoylesoftware.htmlunit.WebClient.loadWebResponseInto(WebClient.java:474)
at com.gargoylesoftware.htmlunit.WebClient.loadDownloadedResponses(WebClient.java:2167)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.doProcessPostponedActions(JavaScriptEngine.java:982)
at com.gargoylesoftware.htmlunit.javascript.JavaScriptEngine.processPostponedActions(JavaScriptEngine.java:1072)
at com.gargoylesoftware.htmlunit.html.DomElement.click(DomElement.java:789)
at com.gargoylesoftware.htmlunit.html.DomElement.click(DomElement.java:732)
- locked <0x0000000670d3fe10> (a com.gargoylesoftware.htmlunit.html.HtmlPage)
at com.gargoylesoftware.htmlunit.html.DomElement.click(DomElement.java:679)
…
at java.lang.Thread.run(Thread.java:744)
Please reopen this jira, this is not fixed in 2.19.
Marc, can you please open a new issue for your case.