Search Results for "js"

...This class decides which URLs should be crawled and handles the downloaded page. shouldVisit function decides whether the given URL should be crawled or not. In the above example, this example is not allowing .css, .js and media files and only allows pages within ics domain. visit function is called after the content of a URL is downloaded successfully. You can easily get the url, text, links, html, and unique id of the downloaded page. You should also implement a controller class which specifies the seeds of the crawl, the folder in which intermediate crawl data should be stored and the number of concurrent threads.

.... ------- Requirements: For the Java HTTP server: - J2SE (SDK for compilation) For the exampe web app: - Apache2 with mod_rewrite and mod_proxy Client-side for the web app: - www browser supporting JS with jQuery --- Setup: - run Server.class from Java server files (java Server) in a terminal/console - unpack example app files under/within your apache2 document root file structure - open index.html (with a www browser, from a www server) - observe terminal output