Search Results for "crawler" - Page 3
Sort By:
Showing 64 open source projects for "crawler"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Earn up to 16% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
1
JCrawler is a perfect cralwing/load-testing tool which is cookie-enabled and follows human crawling pattern (hit/second).
-
2
SmartCrawler is a java-based fully configurable, multi-threaded and extensible crawler, which is able to fetch and analyze the contents of a web site by using dinamically pluggable filters
-
3
WebLoupe is a java-based tool for analysis, interactive visualization (sitemap), and exploration of the information architecture and specific properties of local or publicly accessible websites. Based on web spider (or web crawler) technology.
-
4
Pödznsatch is a open and distributed hypergoogle of love. It is a semantic web application for social networking, word-of-mouth analysis and profiling. The Pödznsatch architecture includes a bot crawler, an inference engine and a query interface.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
-
6
-
7
WebSPHINX is a web crawler (robot, spider) Java class library, originally developed by Robert Miller of Carnegie Mellon University. Multithreaded, tollerant HTML parsing, URL filtering and page classification, pattern matching, mirroring, and more.
-
8
Content Engineering Tools including an XSLT based site rendering system, XSLT Documentation Generator, and Swing based Site Crawler. The tools may be downloaded and used seperately since there are no dependancies between them.
-
9
-
Build Securely on Azure with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
This project aims to be a base for specialized image crawlers. It can download images from a specific website and can be extended to crawler any website. All the the processes are multithread. Accept filters.
-
11
RedditCrawler
Crawls reddit website to pull statistical info.
Reddit Crawler is made to crawl a list of subreddits and get the number of online users. The project will be updated to get more statistical info -
12
Luanium
A Lua-based crawling scripting language and leveraging selenium
...I would put commands in a file or DB to use selenium to interpret the HTML and Javascript. The best would be to have a complete language with conditionals and looping. I'm a java developper and I needed that the crawler to run in a Spring-Boot application. So I decided to use a Lua interpreter in Java to build a crawling tool based on Selenium. The trick here is to add the crawling commands into the Lua interpreter. -
13
Spider is web crawler written in the Java.Based on an Regular expression string the spider parses the internet for web pages matching this string and stores it in an MYSQL database.
-
14
studiMaps is a web based application for visualization and analysis of social networks. It consists of two software components: a web-crawler for getting data and the web based application for visualization.