From: Oscar B. <oba...@um...> - 2019-10-24 19:55:26
|
Hello, I am a novice in using this software and I have read that in using HTMLUnit, there is some association with JUnit (unfortunately, I am not too clear as to what JUnit does in the first place that ties it to HTMLUnit in so many articles on HTMLUnit). Will HTMLUnit not work if I were to try to use its jar files alone for, say, a web-scraping program run on a command-line without using JUnit? Thanks. Oscar |
From: Ronald B. <rb...@rb...> - 2019-10-27 14:17:44
|
Hi Oscar, HtmlUnit has its roots as tool for unit testing (and this is the reason for the name). But there is no restriction - you can use if for web scrapping also. As of today HtmlUnit is a headless browser (written in java) you can use from your java program. And from my knowledge HtmlUnit is today used in many web scrappers. RBRi On Thu, 24 Oct 2019 14:52:12 -0500 Oscar Bastidas wrote: > >Hello, > >I am a novice in using this software and I have read that in using >HTMLUnit, there is some association with JUnit (unfortunately, I am not too >clear as to what JUnit does in the first place that ties it to HTMLUnit in >so many articles on HTMLUnit). > >Will HTMLUnit not work if I were to try to use its jar files alone for, >say, a web-scraping program run on a command-line without using JUnit? >Thanks. > >Oscar > > > >----< Inline text [text-plain-04.txt] >------------------ > > > > >----< Inline text [text-plain-05.txt] >------------------ > >_______________________________________________ >Htmlunit-user mailing list >Htm...@li... >https://lists.sourceforge.net/lists/listinfo/htmlunit-user > > |
From: beshoo <be...@gm...> - 2019-10-27 14:43:34
|
I tested it agents Instagram, and it didn't work. It seems dos not support java script worker's. Please confirm. Regard On Sun, 27 Oct 2019, 4:18 pm Ronald Brill, <rb...@rb...> wrote: > Hi Oscar, > > HtmlUnit has its roots as tool for unit testing (and this is the reason > for the name). > But there is no restriction - you can use if for web scrapping also. > As of today HtmlUnit is a headless browser (written in java) you can use > from your java program. > And from my knowledge HtmlUnit is today used in many web scrappers. > > RBRi > > On Thu, 24 Oct 2019 14:52:12 -0500 Oscar Bastidas wrote: > > > >Hello, > > > >I am a novice in using this software and I have read that in using > >HTMLUnit, there is some association with JUnit (unfortunately, I am not > too > >clear as to what JUnit does in the first place that ties it to HTMLUnit in > >so many articles on HTMLUnit). > > > >Will HTMLUnit not work if I were to try to use its jar files alone for, > >say, a web-scraping program run on a command-line without using JUnit? > >Thanks. > > > >Oscar > > > > > > > >----< Inline text [text-plain-04.txt] >------------------ > > > > > > > > > >----< Inline text [text-plain-05.txt] >------------------ > > > >_______________________________________________ > >Htmlunit-user mailing list > >Htm...@li... > >https://lists.sourceforge.net/lists/listinfo/htmlunit-user > > > > > > > > _______________________________________________ > Htmlunit-user mailing list > Htm...@li... > https://lists.sourceforge.net/lists/listinfo/htmlunit-user > |
From: Oscar B. <oba...@um...> - 2019-10-27 14:50:56
|
Ok, sounds good. Thanks. Oscar Oscar Bastidas, Ph.D. Postdoctoral Research Associate University of Minnesota On Sun, Oct 27, 2019 at 8:59 AM Ronald Brill <rb...@rb...> wrote: > Hi Oscar, > > HtmlUnit has its roots as tool for unit testing (and this is the reason > for the name). > But there is no restriction - you can use if for web scrapping also. > As of today HtmlUnit is a headless browser (written in java) you can use > from your java program. > And from my knowledge HtmlUnit is today used in many web scrappers. > > RBRi > > On Thu, 24 Oct 2019 14:52:12 -0500 Oscar Bastidas wrote: > > > >Hello, > > > >I am a novice in using this software and I have read that in using > >HTMLUnit, there is some association with JUnit (unfortunately, I am not > too > >clear as to what JUnit does in the first place that ties it to HTMLUnit in > >so many articles on HTMLUnit). > > > >Will HTMLUnit not work if I were to try to use its jar files alone for, > >say, a web-scraping program run on a command-line without using JUnit? > >Thanks. > > > >Oscar > > > > > > > >----< Inline text [text-plain-04.txt] >------------------ > > > > > > > > > >----< Inline text [text-plain-05.txt] >------------------ > > > >_______________________________________________ > >Htmlunit-user mailing list > >Htm...@li... > >https://lists.sourceforge.net/lists/listinfo/htmlunit-user > > > > > > |