Re: [Htmlunit-user] trying to load second page from site always goes back to first page

[Guilherme C. H. <gu...@su...>]

Hi,

Debugging the site on firebug i found that when you click the "pagina 2" 
link, the site is reloaded via javascript & json.

So i tried to get the HtmlAnchor and click on it, using this:

List<HtmlAnchor> proximaPagina = page.getByXPath( "//a[contains(@class, 
'js-change-page')]");
HtmlAnchor a = nextPage.get(nextPage.size()-1);
page = a.click(false,false,false,true,true,false);

However, the same page is returned. Shouldn't the click work in this case?

thanks

     guich

Em 24/10/2020 08:17, Guilherme Campos Hazan escreveu:
> Hi,
>
> I'm trying to navigate in the pages of a site. I can successfully 
> download the first page, but when i call the second page (and on), 
> WebClient brings me always the first page. I think that the problem is 
> that this site loads the second page without reloading the page.
>
> This is the code:
>
> java.util.logging.Logger.getLogger("com.gargoylesoftware.htmlunit").setLevel(Level.OFF); 
>
> java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF); 
>
>    System.setProperty("org.apache.commons.logging.Log", 
> "org.apache.commons.logging.impl.NoOpLog");
>    try (WebClient webClient = new WebClient(BrowserVersion.CHROME))
>    {
>       String surl = 
> "https://www.vivareal.com.br/venda/rj/rio-de-janeiro/zona-sul/copacabana/imovel-comercial_comercial/?pagina=2";
> webClient.getOptions().setThrowExceptionOnScriptError(false);
>       HtmlPage page = loadPage((HtmlPage)webClient.getPage(surl));
>       System.out.println(page.asXml();
>    }
>
>    private HtmlPage loadPage(HtmlPage page) throws Exception
>    {
>       String xml;
>       long iniT = System.currentTimeMillis();
>       while (!(xml=page.asXml()).contains("Explore também")/* || 
> xml.length() < 1024*1024*/)
>          try {Thread.sleep(100);} catch (Exception e) {}
>       long endT = System.currentTimeMillis();
>       System.out.println(ind+" arrived "+page.getTitleText()+" with 
> "+xml.length()+" in "+(endT-iniT)+" ms");
>       return page;
>    }
>
> If you paste the surl above in a real browser, you will see that it 
> first loads some data, then without refreshing the page, it loads more 
> data on top of that one.
>
> Any help is greatly appreciated.
>
> thanks
>
>     guich
>
>
> _______________________________________________
> Htmlunit-user mailing list
> Htm...@li...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user