Search Results for "java crawler"
Sort By:
Showing 2 open source projects for "java crawler"
View related business solutions-
Keep company data safe with Chrome EnterpriseMake AI work your way with Chrome Enterprise. Block unapproved sites and set custom data controls that align with your company's policies.
-
Dun and Bradstreet Connect simplifies the complex burden of data managementThe amount, speed, and types of data created in today’s world can be overwhelming. With D&B Connect, you can instantly benchmark, enrich, and monitor your data against the Dun & Bradstreet Data Cloud to help ensure your systems of record have trusted data to fuel growth.
-
1
crawler4j
Open source web crawler for Java
crawler4j is an open source web crawler for Java which provides a simple interface for crawling the Web. Using it, you can setup a multi-threaded web crawler in few minutes. You need to create a crawler class that extends WebCrawler. This class decides which URLs should be crawled and handles the downloaded page. shouldVisit function decides whether the given URL should be crawled or not. -
2
The Java Sitemap Parser can parse a website's Sitemap (http://www.sitemaps.org/). This is useful for web crawlers that want to discover URLs from a website that is using the Sitemap Protocol. This project has been incorporated into crawler-commons (https://github.com/crawler-commons/crawler-commons) and is no longer being maintained.
- Previous
- You're on page 1
- Next
