Re: [Htmlunit-user] Exception thrown by webClient.getPage

[Damon G. <dam...@ho...>]

Hi,
​
As anticipated my attempts to work through your suggestion is creating more effort overhead (i.e. learning some basics) than I can do in a reasonable amount time.  Luckily I have found some help in the form of a book by Kevin Sahin - available here https://www.scrapingbee.com/java-webscraping-book/.  It will take me a little while to try to solve my problem using your suggestion and the help available in this book, which is written at just about my level, if not a bit above it.  My guess is that by the time I have finished I will either have solved my problem or will have a different (set of) problem(s).​
​
So for now I thank you for your help and will get on with what I need to.  BTW if you have any suggestions to supplement this book - which contains some substantial code examples using HTMLUnit - I'm ready to hear them.​
​
Thanks again,


Damon Goodyear
CONFIDENTIALITY and NON-DISCLOSURE OF EMAIL ADDRESS: This email, including its content and the address of the sender, are provided for the use of the recipient only and for the purposes of the subject matter under discussion.   Notwithstanding any other consent that may have been given neither the content of this email nor the address of the sender may be disclosed to third parties, including within the same undertaking, without the prior written permission of the sender.  Where this email has been sent to a general or non-personal email address permission is granted for a reply to be made, on the subject matter only, from the undertaking to whom it is addressed.

________________________________
From: Vasudevan Comandur <vco...@gm...>
Sent: 04 June 2020 12:52
To: htm...@li... <htm...@li...>
Subject: Re: [Htmlunit-user] Exception thrown by webClient.getPage

Hi,

   If you are using Chrome browser, there is an option in the browser to trace the network traffic under more tools->developer tool option
   You can see the request/response and look for the request which gets you the response of interest.
   Call that request from your HTMLUnit program specifying the same parameters and see if you are getting the same response from the server

Regards
 Vasu


On Thu, 4 Jun 2020 at 17:10, Damon Goodyear <dam...@ho...<mailto:dam...@ho...>> wrote:
Hi,

Umm... I'm not sure I know how to do that (i.e. trace through a proxy tool).  I have certainly seen JSON mention in the Warnings given by the getPage function mentioned in my original post.

I don't suppose you could explain your suggestion a little more - and I could then try.

Thanks,



Damon Goodyear
CONFIDENTIALITY and NON-DISCLOSURE OF EMAIL ADDRESS: This email, including its content and the address of the sender, are provided for the use of the recipient only and for the purposes of the subject matter under discussion.   Notwithstanding any other consent that may have been given neither the content of this email nor the address of the sender may be disclosed to third parties, including within the same undertaking, without the prior written permission of the sender.  Where this email has been sent to a general or non-personal email address permission is granted for a reply to be made, on the subject matter only, from the undertaking to whom it is addressed.

_______________________________________________
Htmlunit-user mailing list
Htm...@li...<mailto:Htm...@li...>
https://lists.sourceforge.net/lists/listinfo/htmlunit-user<https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.sourceforge.net%2Flists%2Flistinfo%2Fhtmlunit-user&data=02%7C01%7C%7Cd97a06a977984d7253a408d8087ddf01%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637268683986628516&sdata=QM5mywTCKTuFuKtILJxfghGZ5TBmAnN4C%2FhEyVlrX0E%3D&reserved=0>