Menu
▾
▴
#1967 Clicking button has stopped working and returns same web page
Until recently, I have been running HtmlUnit 2.25. It was throwing up rhino errors but still allowed my code to click the edit button in a Google Sites web page to bring up the editable version of the page. My code has recently stopped working. Now when I call click() to click the edit button, the same web page is returned (easy to verify since the returned web page still contains the edit button and not the save button of the editable web page).
I have switched to HmtlUnit 2.31. The good news is the rhino errors have gone. But click() is still not advancing to the editable web page.
The problem doesn't seem likely to be a Javascript error, since once the right element is obtained I don't think Javascript is involved in the actual click() process.
Here is my code:
:::java
// Set up the web client.
WebClient webClient = new WebClient(BrowserVersion.CHROME);
// Read in cookies file so that the Google Sites server will recognise
// me as logged in and return the logged in web page in the following
// getPage call (code not shown here as this all works fine).
// Get logged in web page.
HtmlPage loggedInPage= webClient.getPage("https://sites.google.com/site/[my-web-page-at-google-sites]");
// Wait 2 secs just in case time is needed to execute Javascript.
webClient.waitForBackgroundJavaScriptStartingBefore(2000);
// Get the element (edit button) to click.
HtmlElement element loggedInPage.getElementById("edit-start-btn");
// Check that element obtained is the correct edit button and all is
// looking good for clicking it.
System.out.println("Element about to be clicked is: + element);
// Click the edit button to get the editable web page.
HtmlPage editablePage = element.click();
// Have a look at the returned web page to see if it is the editable page.
System.out.println(editablePage.asXml());
Here is the web page code snippet containing the button I want to click:
<div role="button" id="edit-start-btn" class="goog-inline-block jfk-button jfk-button-standard jfk-button-disabled" aria-disabled="true">
<span id="sites-collaborator-bar-edit-page-icon" class="sites-camelot-icon">
</span>
</div>
My code finds the element on the web page OK since the println that outputs details about the element about to be clicked corresponds correctly with the edit button shown in this web page code snippet above:
Element about to be clicked is: HtmlDivision[<div role="button" id="edit-start-btn" class="goog-inline-block jfk-button jfk-button-standard jfk-button-collapse-right collaborator-btn-collapse-right" aria-disabled="false" aria-label="Edit page" tabindex="0">]
Why does my code fail, and why should it have previously worked OK in HtmlUnit 2.25 and then suddenly stopped working?
I notice that the time now taken to perform the click() is at most a second or apparently instantaneous. When the code was working, it took a second or two. Could this indicate that HtmlUnit is not even going to the Google Sites server, perhaps returning the cached version of the same page instead? One thought I had but probably not relevant: could it be a focus issue? I know HtmlUnit has changed for text input in forms, now requiring the focus to be shifted to the form which wasn't the case before. Does the focus have to be shifted to a button before clicking it will work?
Many thanks if you can help.
I find HtmlUnit really useful in the work I do. It's an awesome tool and you guys do a fantastic job in developing and supporting it.
Having done some more testing, I realise that the problem is probably Javascript-related.
I tried all the other BrowserVersions to CHROME (which I have been using up to now) to see if any might work. All fail, except for INTERNET_EXPLORER which works some of the time.
My Java application performs a number of editing cycles on a Google Sites web page. A cycle consists of getting the editable web page, making an edit and saving the result. Using INTERNET_EXPLORER works for the first editing cycle but fails for second and subsequent cycles.
To make my code reliable and minimise the possibility of any HtmlUnit-related problems occurring (chance of memory leakage, etc), I use a new WebClient each editing cycle. At the start of each cycle, I instantiate a WebClient. At the end of that cycle, I tidy up as thoroughly as I can by calling close() to close all open windows, calling getCache().clear() to clear the cache, and nulling the WebClient instance that is no longer needed.
In the first editing cycle when using INTERNET_EXPLORER, getPage returns the element that acts as the edit button as:
After waiting two seconds for Javascript to render the page, this element changes to:
The fact that the attribute 'aria-disabled="true"' has changed to 'aria-disabled="false"' seems to be the indicator that clicking the element is now going to work. Sure enough, performing click() on the element in this state works fine, returning the editable web page with the save button.
On the second and subsequent editing cycle, getPage initially returns the same element as it did in the first cycle. But however long my Java application waits (I tried 90 secs), the element does not change. Understandably, performing click() on the element while it is in this state fails to bring up the editable web page.
When I try using CHROME as the BrowserVersion, getPage on the first editing cycle returns:
After waiting two seconds for Javascript to render the page, this element changes to:
All looks good yet despite the element seeming to have been rendered (attribute 'aria-disabled="true"' changing to 'aria-disabled="false"'), performing click() on the element fails.
On the second and subsequent editing cycle with CHROME, just like with INTERNET_EXPLORER the element never changes (ie, 'aria-disabled="true"' never changes to 'aria-disabled="false"'). And of course performing click() fails.
Can anyone suggest what could cause this observed behaviour, since it might point the way to why my code isn’t working?
Things I don't understand:
1/ When using INTERNET_EXPLORER, why does HtmlUnit handle getting the web page differently after the first cycle? No instantiated HtmlUnit code remains between cycles, so how is the second (failing) cycle any different from the first (working) cycle? The only common factor is the continuing instantiation of my Java application. If I stop my application and restart it, I get the same behaviour again: the newly instantiated application doing the first editing cycle OK but failing from then on. Somehow there must be some ‘memory’ that getting the editable web page has been done before, with this 'memory' somehow being the cause of the problem. But that does not seem possible since all HtmlUnit-related objects are built fresh each cycle!?
2/ CHROME exhibits the same difference in behaviour between first and subsequent cycles as INTERNET_EXPLORER does. Yet although the element for the edit key does change in the first cycle (which would seem to indicate that click() is going to work since it did for INTERNET_EXPLORER), the first cycle fails. Something else must be going on!?
3/ I have had similar problems with clicking buttons in the past. For some reason, only the occasional HtmlUnit release seems to work for me, the last release that reliably worked being 2.25. Since 2.25 I haven't managed to get any more recent release to work. Why do most releases fail for what I want to do, and what is it that makes the occasional one work? There must be something special about these working versions!?
Hi Ron,
Just wondering if there is anything more I can add to my bug description to help you guys pinpoint the problem and possibly provide a fix?
The fact that I can get my code to work consistently for some of the time suggests to me that a relatively easy solution, or at least a workaround, might be possible. The trouble is I just don't know enough about the inner workings of HtmlUnit to know what that fix/workaround might be. I am hoping that by explaining the behaviour of the bug in my last post, it might suggest to you the type of problem I am facing.
Any thoughts or suggestions you have would be really helpful, as having my code not working is causing me a lot of problems. Even if you are only able to say that the bug isn't easy to fix, or is unlikely to be fixed in the near future, would help as then I can at least plan what I need to do next.
Many thanks!
Martin
Will have a look at this during the next days.
Thank you so much. Greatly appreciated!
Sorry, there was something else more important on my desk, but now i will have a look at your problem.
Let me know if I can provide any more info to help you. Thanks for keeping me in the loop.
Can you please
Last edit: RBRi 2018-07-27
Tried all your suggestions but unfortunately none worked.
I assumed by 'latest snapshot' you meant the new release of HtmlUnit 2.32 that has just gone live. Would generating a snapshot be any different from 2.32?
In all tests before calling element.click(), element.isDisplayed() returned true.
Could not try checking for element.isDisabled() since this method call was not available. Why should this be? My element is an instance of HmtlElement. Should it be something else since I can't find the isDisabled() method?
Tried HtmlUnit 2.32 using all browsers (Chrome, IE, Edge, Firefox52, Firefox60) with element.click(false, false, false, true, true, false) . Apart from Edge, all worked fine on my first edit cycle (as discussed in previous post) then failed in subsequent edit cycles.
Edge failed on the first cycle. As part of my edit cycle, I call click() again if the returned web page doesn't have the editable content. On this second call with element.click(false, false, false, true, true, false) I got the following exception:
Let me know what you suggest next and if I can do some more testing to help you track down the problem.
Thanks for finding the time to work on this!
Hi Martin,
can you please contact me by private mail. I fear i have do do some debugging and i need some hints from you to get your cookie magic working.
Hi Gents,
I am having the same issue since a couple of days and saw your tickeet now.
Can I ask if you could find something already?
I could easily reproduce it with Google and cklicking the "search" button.
But I tested it with 2.3.1 and 2.3.3 please...
Last edit: lin83 2019-02-06
I made more tests now and found the following interesting:
If I use "firstXPath" to find the element, then it's all working.
But when I use ID or name, then it is failing to renew the page... Even the element is detected as HtmlSubmitInput button.
Do you want me to create a new ticket with some logs?
More tests:
Still let me know if you require some more informations, Happy to deliver them.
Hi lin83, thanks for your comments. RBRI suggested I try using element.click(false, false, false, true, true, false) instead of element.click() but it didn't fix my problem (as I explained above). So you found it worked for you?
What version are you using - 2.33? My problem was that click() stopped working after working OK the first time. Are you finding that by using element.click(false, false, false, true, true, false) you never have a problem at any time?
Another problem I am having is logging into Google. I am only successful if I switch Javascript off for the webClient and log in at Google's old login page that does not require a Javascript-enabled browser. If you are editing a Google website page like me, you must have managed to log into Google too and I wonder how you have done this. Are you managing to log into Google at their normal, Javascript-enabled login page? If so, can you tell me how you do it?