Spatie Crawler

Spatie Crawler is a PHP library that allows developers to crawl websites and extract information efficiently. It can be used for web scraping, link checking, or automated testing of web pages. The library is simple to use and supports customizable crawling strategies, including controlling crawl depth and handling redirects. It’s suitable for building crawlers that navigate large or dynamically generated websites.

Features

Easily crawl websites to extract information
Supports setting crawl depth and controlling the number of pages
Handles redirects and follows links intelligently
Allows filtering of URLs using custom rules
Supports asynchronous crawling for better performance
Customizable callbacks for processing crawled pages

Project Samples

Project Activity

See All Activity >

License

MIT License

Follow Spatie Crawler

Spatie Crawler Web Site

Other Useful Business Software

Custom VMs From 1 to 96 vCPUs With 99.95% Uptime

General-purpose, compute-optimized, or GPU/TPU-accelerated. Built to your exact specs.

Live migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.

Try Free

Rate This Project

User Reviews

Be the first to post a review of Spatie Crawler!

Additional Project Details

Programming Language

PHP

Related Categories

PHP Libraries

Registered

2025-05-16

Similar Business Software

DHTMLX

DHTMLX is a JavaScript UI library that provides a set of highly customizable and flexible components for building modern and responsive web applications. The library includes more than 30 UI components, such as Gantt, Scheduler, Kanban, diagrams, charts, grids, spreadsheets, calendars, trees,...

See Software
Webix

JavaScript UI library and framework for speeding up web development. JS Framework for cross-platform web Apps development 102 UI widgets and feature-rich CSS / HTML5 JavaScript controls. Save at least 3000+ development hours by using ready-made widgets and UI controls. Develop Web UI 30% faster....

See Software
JsPHP

The free open-source JsPHP library. The JsPHP website is a 100% free community resource that provides a collaborative platform and web-based Integrated Development Environment (IDE) for building an open-source JavaScript library called JsPHP that provides an implementation of the PHP API for...

See Software
FusionCharts

FusionCharts is a powerful and easy-to-use JavaScript charting library that helps developers to add interactive charts and data visualizations to their web and mobile applications. With 100+ chart types, including column, bar, line, area, pie, doughnut, scatter, bubble, and more, it's easy to...

See Software
Kendo UI

Kendo UI is the ultimate collection of JavaScript UI components with libraries for jQuery, Angular, React, and Vue. Quickly build eye-catching, high-performance, responsive web applications—regardless of your JavaScript framework choice. Easily add advanced JavaScript components into your...

See Software
Sencha Ext JS

Sencha Ext JS is a comprehensive JavaScript application framework for building feature-rich, cross-browser-compatible web and mobile applications. It includes a library with 140+ high-performance customizable components, a set of tools and a powerful UI Framework. Key Features Rich UI...

See Software

Report inappropriate content

Spatie Crawler

An easy to use, powerful crawler implemented in PHP

Get an email when there's a new version of Spatie Crawler

Features

Project Samples

Project Activity

Categories

License

Follow Spatie Crawler

User Reviews

Additional Project Details

Programming Language

Related Categories

Registered