OpenWebSpider

OpenWebSpider is an Open Source multi-threaded Web Spider (robot, crawler) and search engine with a lot of interesting features!

Project Samples

Project Activity

See All Activity >

License

MIT License

Follow OpenWebSpider

OpenWebSpider Web Site

Other Useful Business Software

Gen AI apps are built with MongoDB Atlas

Build gen AI apps with an all-in-one modern database: MongoDB Atlas

MongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.

Start Free

Rate This Project

User Ratings

4.2 out of 5 stars

★★★★★

★★★★

★★★

★★

★

ease 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 3 / 5

features 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 4 / 5

design 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 4 / 5

support 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 3 / 5

User Reviews

Filter Reviews:

All

gnulinuxaholic Posted 2015-09-09

Congratulations, very complete search tool! Guy, please, how to configure the login/password for proxy client/server?
docdaven Edited 2015-05-06

as with a lot of these projects, they don't run on windows without errors such as 'node' is not recognized as an internal or external command popping up. But I can't complain, it's free, but the average person will have to spend a few hours figuring out how to get it to work or reconfiguring windows to run scripts.
sawiolka Posted 2014-12-04

Well done
tfileme Posted 2013-05-20

Nice work ! Great project.

1 user found this review helpful.

Additional Project Details

Operating Systems

BSD, Linux, Windows

User Interface

Web-based

Programming Language

C#, JavaScript, PHP

Database Environment

MySQL, Other network-based DBMS

Related Categories

C# Search Engines, C# Web Scrapers, PHP Search Engines, PHP Web Scrapers, JavaScript Search Engines, JavaScript Web Scrapers

Registered

2004-07-12

Similar Business Software

Lobstr.io

Get the data you need. Lobstr is a web scraping software that offers ready-made no-code solution to collect data from websites. Users can extract information from sources like social media, e-commerce sites, and search engines. Best no-code scrapers are: * Google Maps Search...

See Software
Dekoo

Dekoo is the Asia's independent global search engine company offering Unique search, Unbiased results on the web having its own crawler, indexing, search and ranking algorithms along with AI/ML technologies with Tier 4 Datacenters and Content Delivery Network around the world. It aims to...

See Software
Piloterr

Piloterr API handles crawlers and rotates proxies for you. Piloterr is a software platform that enables forward-thinking companies to leverage the full potential of the web. Piloterr gives you the tools and know-how to grow your business faster. We are always expanding our code and...

See Software
Crawl4AI

Crawl4AI is an open source web crawler and scraper designed for large language models, AI agents, and data pipelines. It generates clean Markdown suitable for retrieval-augmented generation (RAG) pipelines or direct ingestion into LLMs, performs structured extraction using CSS, XPath, or...

See Software
Oxylabs

Oxylabs is a market leader in web intelligence with enterprise-grade, ethical, and compliant solutions. Its proxy infrastructure spans one of the largest global networks, offering residential, ISP, mobile, datacenter, and dedicated datacenter proxies, along with Web Unblocker – an AI-driven...

See Software
Infospace

InfoSpace is a leading provider of white label search and monetization solutions. We serve a network of more than 100 web publishers globally, offering core search solutions as a proven means to monetize audience via a search-based advertising revenue stream. Our technology selects from the top...

See Software