Audience
SEO
About Website Crawler
Website Crawler is a cloud-based On Page SEO checker which you can use to find broken internal/external links, missing title, meta description, alt, h1, h2, h3, h4, h5 tags, thin content, slow-loading pages, etc on the pages of your website.
Pricing
Starting Price:
0
Pricing Details:
We have a paid plan that costs $10 per month.
Free Version:
Free Version available.
Integrations
Company Information
Website Crawler
Founded: 2019
websitecrawler.org
Videos and Screen Captures
Screaming Frog SEO Spider
The Screaming Frog SEO Spider is a website crawler that helps you improve onsite SEO, by extracting data & auditing for common SEO issues. Download & crawl 500 URLs for free, or buy a license to remove the limit & access advanced features. The SEO Spider is a powerful and flexible site crawler, able to crawl both small and very large websites efficiently while allowing you to analyze the results in real-time. It gathers key onsite data to allow SEOs to make informed decisions. Crawl a website instantly and find broken links (404s) and server errors. Bulk export the errors and source URLs to fix, or send to a developer. Find temporary and permanent redirects, identify redirect chains and loops, or upload a list of URLs to audit in a site migration. Analyze page titles and meta descriptions during a crawl and identify those that are too long, short, missing, or duplicated across your site.
Learn more
SiteAnalyzer
SiteAnalyzer is a web crawler tool meant to scan URLs and perform a fast SEO analysis of a website. Use it to fix technical issues and improve SEO in the shortest period. Scan your entire website and get detailed information about each page. SiteAnalyzer will help you identify and fix both technical and SEO errors! SiteAnalyzer provides a set of tools for quick and effective technical website audit and SEO optimization. Scan URLs to detect broken links and non-existent pages. Check 301 and 302 redirects on a website (temporary and permanent redirects). Optimize title, keywords, and description tags and combat duplicates. Search the internal and external links on your website. Calculate the Google PageRank of each page on your website. Unlike other crawlers, SiteAnalyzer allows to save projects as a list instead of accessing them one by one. Scan websites of any size due to modest system requirements.
Learn more
Netpeak Spider
Netpeak Spider is an SEO crawler for a day-to-day SEO audit, fast issue check, comprehensive analysis, and website scraping.
This tool allows you to:
* Spot 100+ issues of your website optimization.
* Check 80+ key on-page SEO parameters.
* Calculate internal PageRank to improve website linking structure.
* Analyze all incoming and outgoing internal links.
* View page source and HTTP headers.
* Generate sitemaps: XML, Image and HTML.
* Adjust Netpeak Spider to your own requirements using crawling modes for the entire website, the URL list or XML Sitemap.
* Set custom rules to crawl either the entire website or its certain part
* Consider indexation instructions (Robots.txt, Meta Robots, X-Robots-Tag, Canonical)
* Perform custom search of source code/text using 4 types of search.
* Avoid duplicate content: Pages, Titles, Meta Descriptions, H1 Headers, etc.
* Spot issues with redirects.
* Overview panel for fast SEO audit with special status codes which show website
Learn more
TechSEO360
TechSEO360 is an all-in-one technical SEO crawler software tool which can :
- Fix broken links, broken redirects and broken canonical references.
- Find pages with thin content, duplicate titles, duplicate headers, duplicate meta and similar content.
- Analyze keywords across single pages or entire websites.
- Create all kinds of sitemaps including HTML, XML, image and video including hreflang information.
- Integrate with various 3d party data exports including Apache logs, Google Search Console and more. Data from those sources can then be combined with what TechSEO360 has collected to generate custom reports which can be exported to CSV and Excel.
- Crawl very large websites.
- Include searching Javascript code for links.
- Use the software in AJAX mode for websites that require this.
- Configure the crawler with both limit-to and exclude filters separately for analysis and output.
- Use command line interface to schedule and automate most of the work.
Learn more
You Might Also Like
SKUDONET ADC, operates at the application layer, efficiently distributing network load and application load across multiple servers. This not only enhances the performance of your application but also ensures that your web servers can handle more traffic seamlessly.
Product Details
Platforms Supported
SaaS
Training
Documentation
Support
Phone Support
Website Crawler Frequently Asked Questions
Website Crawler Product Features
SEO
Dashboard
A/B Testing
Artificial Intelligence (AI)
Auditing
Competitor Analysis
Content Management
Google Analytics Integration
Keyword Research Tools
Keyword Tracking
Link Management
Localization
Mobile Search Tracking
Rank Tracking
Revenue Management
User Management
Website Crawler Reviews
Write a Review-
Probability You Would Recommend?1 2 3 4 5 6 7 8 9 10
"Best Web Crawler SEO" Edited 2020-07-08
Pros: A website crawler is an excellent SEO tool to find broken links, tag errors, meta, duplicate content, etc. It's a free crawler and it does it work very fast and accurately. Very easy to use in WordPress websites.
Cons: It's an outstanding SEO tool to analyze web pages.
Overall: Website Crawler is a terrific SEO tool. Overall I used this tool for my website. It basically moves a spider bot and crawls through the website. It finds broken links, alt tags, etc. It's an outstanding tool.
Read More...
- Previous
- You're on page 1
- Next