Crawler is a bare-bones spider designed to quickly and effectively build an index of all files and pages on a given Web site as well as the link relationship (both incoming and outgoing) between each page. More open source at https://github.com/fcc.
License
GNU General Public License version 2.0 (GPLv2)Follow Crawler
Other Useful Business Software
$300 Free Credits to Build on Google Cloud
Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of Crawler!