Search Results for "web crawler source code"
Sort By:
Showing 6 open source projects for "web crawler source code"
View related business solutions-
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Spider
High-performance Rust web crawler and scraper for large-scale data
Spider is a high-performance web crawler and web scraping library written in Rust that enables developers to crawl and index websites efficiently. It focuses on speed, concurrency, and reliability by using asynchronous and multi-threaded processing to handle large volumes of web pages. It can rapidly crawl websites to collect links, retrieve page content, and extract structured information from HTML documents. Spider can operate concurrently across many pages, allowing it to gather large... -
2
ffsend
Easily and securely share files from the command line
Easily and securely share files and directories from the command line through a safe, private and encrypted link using a single simple command. Files are shared using the Send service and may be up to 1GB. Others are able to download these files with this tool, or through their web browser. All files are always encrypted on the client, and secrets are never shared with the remote host. An optional password may be specified, and a default file lifetime of 1 (up to 20) download or 24 hours is... -
3
Russh
Rust SSH client & server library
Russh provides a Rust library for implementing SSH clients and servers with a modern, async-friendly design. It exposes building blocks for authentication, channel management, port forwarding, and key handling, allowing you to embed SSH functionality directly into Rust applications. The API is designed to be explicit and composable, making it possible to implement custom behaviors like reverse tunnels, interactive shells, and service multiplexing. Because performance and safety are central,... -
4
AeroFTP
AeroFTP is a Cross-platform desktop client for FTP, SFTP, WebDAV, S3
AeroFTP is a cross-platform file transfer client that goes beyond traditional FTP. Connect to 25+ protocols, FTP/FTPS, SFTP, WebDAV, S3, Google Drive, Dropbox, OneDrive, MEGA, Box, pCloud, Azure, Filen, and more from a single interface. Security-first: AeroVault v2 encrypted containers (AES-256-GCM-SIV), Cryptomator support, and zero telemetry. Built-in AeroAgent AI assistant with 19 providers and 47 tools for file operations and workflow automation. Includes Monaco editor,... -
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
5
Rouille
Web framework in Rust
Rouille is a micro-web-framework library. It creates a listening socket and parses incoming HTTP requests from clients, then gives you the hand to process the request. Rouille was designed to be intuitive to use if you know Rust. Contrary to express-like frameworks, it doesn't employ middlewares. Instead, everything is handled in a linear way. Concepts closely related to websites (like cookies, CGI, form input, etc.) are directly supported by rouille. More general concepts (like database... -
6
binserve
A fast production-ready static web server with TLS (HTTPS)
A fast static web server with TLS (HTTPS), routing, hot reloading, caching, templating, and security in a single binary you can set up with zero code. Binserve is designed to be performant, this is thanks to Actix-Web - one of the fastest web frameworks out there and DashMap for handling routes and cache storage. Binserve is cross-platform and portable to any major operating system, like it can run on your Android phone! Routing is simply matching a URI path to a file or a directory in a...
- Previous
- You're on page 1
- Next
