Linkcrawler LinkCrawler

Capable to "Crawl" a site and return a report of all links from it

Brought to you by: cumanzor

Commit [r49] Maximize Restore History

- Adding new HTTPConnector as a centralized way to perform HTTP connections

- Fixing isCrawlable method in order to prevent failed crawl attempts
- Capturing Content-Type during crawling in order to achieve better control
- Fixing Sitemap XML section with more accurate info

Authored by: cumanzor 2013-11-06

Browse code at this revision

Parent: [r48]

Child: [r50]

changed	/trunk/LinkCrawler/src/main/java/linkcrawler/LinkCrawlerMain.java
added	/trunk/LinkCrawler/src/main/java/linkcrawler/connectors
added	/trunk/LinkCrawler/src/main/java/linkcrawler/connectors/HTTPConnector.java
changed	/trunk/LinkCrawler/src/main/java/linkcrawler/datatypes/LinkStatus.java
changed	/trunk/LinkCrawler/src/main/java/linkcrawler/datatypes/URLObject.java
changed	/trunk/LinkCrawler/src/main/java/linkcrawler/logic/htmlUnitEngine/HTMLUnitSitemapVerificator.java
changed	/trunk/LinkCrawler/src/main/java/linkcrawler/logic/htmlUnitEngine/HtmlUnitCrawler.java

/trunk/LinkCrawler/src/main/java/linkcrawler/LinkCrawlerMain.java Diff Switch to side-by-side view

/trunk/LinkCrawler/src/main/java/linkcrawler/connectors

Directory.

/trunk/LinkCrawler/src/main/java/linkcrawler/connectors/HTTPConnector.java Diff Switch to side-by-side view

/trunk/LinkCrawler/src/main/java/linkcrawler/datatypes/LinkStatus.java Diff Switch to side-by-side view

/trunk/LinkCrawler/src/main/java/linkcrawler/datatypes/URLObject.java Diff Switch to side-by-side view

/trunk/LinkCrawler/src/main/java/linkcrawler/logic/htmlUnitEngine/HTMLUnitSitemapVerificator.java Diff Switch to side-by-side view

/trunk/LinkCrawler/src/main/java/linkcrawler/logic/htmlUnitEngine/HtmlUnitCrawler.java Diff Switch to side-by-side view