Search Results for "crawler" - Page 9
Sort By:
Showing 245 open source projects for "crawler"
View related business solutions-
Stop Storing Third-Party Tokens in Your DatabaseRolling your own OAuth token storage can be a security liability. Token Vault securely stores access and refresh tokens from federated providers and handles exchange and renewal automatically. Connected accounts, refresh exchange, and privileged worker flows included.
-
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
zSearch is a simple python based crawler and search engine. Raw HTML are stored in bzip2 archives, the index is created using pylucene, and twsited is used to provide internal http server. Results are sent back as XML over HTTP.
-
2
WebNews Crawler is a specific web crawler (spider, fetcher) designed to acquire and clean news articles from RSS and HTML pages. It can do a site specific extraction to extract the actual news content only, filtering out the advertising and other cruft.
-
3
Crawler.NET is a component-based distributed framework for web traversal intended for the .NET platform. It comprises of loosely coupled units each realizing a specific web crawler task. The main design goals are efficiency and flexibility.
-
4
Course Crawler is an application to compile term-definition pair from multiple web glossaries into a centralized, stable, and searchable location.
-
Earn up to 16% annual interest with Nexo.Generate interest, borrow against your crypto, and trade a range of cryptocurrencies — all in one platform. Geographic restrictions, eligibility, and terms apply.
-
5
Crawl-By-Example runs a crawl, which classifies the processed pages by subjects and finds the best pages according to examples provided by the operator. Crawl-By-Example is a plugin to the Heritrix crawler, and was done as a part of GSoC06 program.
-
6
GronoSpy is a WWW crawler which tries to extract knowledge based on the data from grono.net - a community portal.
-
7
SNT is a search engine for SMB and FTP shares with crawler running on Win32. Web interface is provided for searching files and browsing shares contents. Also provided shared films list with users rates and comments.
-
8
-
9
J-Obey is a Java Library/package, which allows people writing their own crawlers to have a stable Robots.txt parser, if you are writing a web crawler of some sort you can use J-Obey to take out the hassle of writing a Robots.txt parser/intrepreter.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
Light network file search engine, is a crawler of FTP servers and SMB shares (Windows shares and UNIX systems running Samba). WWW Perl(Mason) interface is provided for searching files.
-
11
Very simple single-thread crawler for web (http) written in perl. Supports links following rules and collections to grab information of visited pages (regexp based).
-
12
A configurable knowledge management framework. It works out of the box, but it's meant mainly as a framework to build complex information retrieval and analysis systems. The 3 major components: Crawler, Analyzer and Indexer can also be used separately.
-
13
Larbin is a Web crawler intended to fetch a large number of Web pages, it should be able to fetch more than 100 millions pages on a standard PC with much u/d. This set of PHP and Perl scripts, called webtools4larbin, can handle the output of Larbin and p
-
14
JCrawler is a perfect cralwing/load-testing tool which is cookie-enabled and follows human crawling pattern (hit/second).
-
15
Fast File Search is a crawler of FTP servers and SMB shares (Windows shares and UNIX systems running Samba). WWW interface is provided for searching files. FFS is similar to FemFind but optimized for speed.
-
16
HtmlTester is a software to test any active/passive web sites. It has Crawler to collect all urls and forms in config file - after that YOU have to type the data to test how your server/pages will react. Run the Tester and see results in the console.
-
17
SmartCrawler is a java-based fully configurable, multi-threaded and extensible crawler, which is able to fetch and analyze the contents of a web site by using dinamically pluggable filters
-
18
DSE, Distributed Search Engine, is highly scalable open source component-based search engine for crawling and searching of the Web. It incapsulates a crawler, indexer, query manager, web front-end for query.
-
19
WebLoupe is a java-based tool for analysis, interactive visualization (sitemap), and exploration of the information architecture and specific properties of local or publicly accessible websites. Based on web spider (or web crawler) technology.
-
20
WWW Universal Tester is a Java application designed to gather information about WWW. She works as a spider (robot, crawler) and collets information about size of files used on the web, structure of connections between pages, on so on.
-
21
The Jobcrawler search engine is a research project in order to index the available applications on the internet. Our mission is to really help people who seek a job or employee on a one to one basis and rule mediators (job agencies) out.
-
22
Nomad is tiny but efficient search engine and web crawler. This works very good for searching with in the set of corporate websites on internet and/or intranet's HTML documents or knowledge repositories.
-
23
-
24
Web Textual eXtraction Tools C++ Parallel web crawler, noun phrase idenification, Multi-lingual Part of Speech Tagging, Tarjan's Algorithm, Co-RelationShip Mappings...
-
25
Larbin is an HTTP Web crawler with an easy interface that runs under Linux. It can fetch more than 5 million pages a day on a standard PC (with a good network).
