Search Results for "web crawler source code"
Sort By:
Showing 39 open source projects for "web crawler source code"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
1
Every Code
Local AI coding agent CLI with multi-agent orchestration tools
Every Code (often referred to simply as Code) is a fast, local AI-powered coding agent designed to run directly in the terminal environment. It is a community-driven fork of the Codex CLI, with a strong emphasis on improving real-world developer ergonomics and workflows. Every Code enhances the traditional coding assistant model by introducing multi-agent orchestration, allowing multiple AI agents to collaborate, compare solutions, and refine outputs in parallel. It supports integration with... -
2
Forge Code
AI enabled pair programmer for Claude, GPT, O Series, Grok, Deepseek
Forge is a modern, open-source tool that brings AI-powered code assistance directly into your terminal workflow, effectively turning your shell into a “pair programmer”, without ever leaving your development environment. Written in Rust (with a command-line interface), Forge integrates with your existing shell (bash, zsh, fish, etc.) or IDE-agnostic workflows, allowing you to interact with your codebase, command-line tools, and version control as usual, but with the added support of large language models (LLMs) to help with code generation, refactoring, bug fixing, code review, and even design advice. ... -
3
Spider
High-performance Rust web crawler and scraper for large-scale data
Spider is a high-performance web crawler and web scraping library written in Rust that enables developers to crawl and index websites efficiently. It focuses on speed, concurrency, and reliability by using asynchronous and multi-threaded processing to handle large volumes of web pages. It can rapidly crawl websites to collect links, retrieve page content, and extract structured information from HTML documents. Spider can operate concurrently across many pages, allowing it to gather large... -
4
cc-switch
A cross-platform desktop All-in-One assistant tool for Claude Code
cc-switch is a cross-platform desktop assistant designed to unify and streamline the management of multiple AI coding tools such as Claude Code, Codex, OpenCode, and Gemini CLI within a single interface. The application focuses on improving developer productivity by providing centralized configuration, provider switching, and workflow orchestration across different AI backends. Built as a modern desktop app using Tauri and web technologies, it enables users to manage credentials, sessions,... -
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
5
cargo-crev
A cryptographically verifiable code review system for the cargo
...Increase the trustworthiness of your own code. Build a web of trust of other reputable users to help verify the code you use. Static binaries are available from the releases page. Crev is a system for verifying the security and reliability of dependencies based on collaborative code reviews. Crev users review the source code of packages/libraries/crates and share their findings with others. -
6
Biome
A toolchain for web projects, aimed to provide functionalities
Biome formats and lints your code in a fraction of a second. Biome supports JavaScript, TypeScript, JSON, and CSS. It aims to support all main languages of modern web development. Biome has sane defaults and requires minimal configuration. Biome helps you as much as possible by displaying detailed and contextualized diagnostics. Biome unifies functionality that has previously been separate tools. Building upon a shared base allows us to provide a cohesive experience for processing code,... -
7
Solana
Web-scale blockchain for fast, secure, scalable, decentralized apps
Solana is the fastest blockchain in the world and the fastest-growing ecosystem in crypto, with thousands of projects spanning DeFi, NFTs, Web3 and more. Integrate once and never worry about scaling again. Solana ensures composability between ecosystem projects by maintaining a single global state as the network scales. Never deal with fragmented Layer 2 systems or sharded chains. Solana's scalability ensures transactions remain less than $0.01 for both developers and users. Solana is all... -
8
Sycamore
A library for creating reactive web apps in Rust and WebAssembly
A reactive library for creating web apps in Rust and WebAssembly. Sycamore harnesses the full power of Rust via WebAssembly, giving you full control over performance. Write code that feels natural. Everything is built on reactive primitives without a cumbersome virtual DOM. No JavaScript. Had enough of JavaScript? So have we. Create apps using Sycamore without touching a single line of JS. -
9
SWC
Rust-based platform for the Web
...It's used by tools like Next.js, Parcel, and Deno, as well as companies like Vercel, ByteDance, Tencent, Shopify, and more. SWC can be used for both compilation and bundling. For compilation, it takes JavaScript / TypeScript files using modern JavaScript features and outputs valid code that is supported by all major browsers. SWC is 20x faster than Babel on a single thread and 70x faster on four cores. SWC can be downloaded and used as a pre-built binary, or built from the source. SWC (stands for Speedy Web Compiler) is a super-fast TypeScript / JavaScript compiler written in Rust. It's a library for Rust and JavaScript at the same time. ... -
AI-powered service management for IT and enterprise teamsGive your IT, operations, and business teams the ability to deliver exceptional services—without the complexity. Maximize operational efficiency with refreshingly simple, AI-powered Freshservice.
-
10
Leptos
Build fast web applications with Rust
Leptos is a full‑stack Rust framework for building web apps that supports client-only SPAs, SSR/MPR rendered on the server, and hydrated progressive apps. It offers fine‑grained reactivity, server functions, and cohesive front‑end/backend code using shared Rust abstractions. -
11
IronClaw
IronClaw is OpenClaw inspired but focused on privacy & security
IronClaw is a security-first, open-source personal AI assistant built in Rust and designed to keep your data fully under your control. It operates on the principle that your AI should work for you, not external vendors, ensuring all data is stored locally, encrypted, and never shared. The platform emphasizes transparency, offering auditable code with no hidden telemetry or data harvesting. -
12
modrinth
The Modrinth monorepo containing all code which powers Modrinth
modrinth is the primary open-source monolithic repository that contains the backend, web interface, and core infrastructure powering the Modrinth mod distribution platform. The project consolidates the codebase used to operate the Modrinth website, its APIs, and related services that allow users to discover, publish, and manage Minecraft mods, plugins, and modpacks. The platform is designed to support a modern ecosystem for mod distribution, providing developers with tools to upload and... -
13
Shuttle Framework
Deploy Rust apps with a single Cargo command
...And we support multiple providers. A completely free and open-source project. Start deploying your apps with no strings attached. Deploy new versions as quickly as running an incremental build, all with zero downtime. Take your code to full-featured cloud infrastructure in under a minute. Don't take our word for it, see it for yourself. Build any web service with a fully managed database using Rocket and sqlx. -
14
Rocket
A web framework for Rust
Rocket is a web framework for Rust that makes it simple to write fast, secure web applications without sacrificing flexibility, usability, or type safety. Handling forms is simple and easy. Simply derive FromForm for your structure and let Rocket know which parameter to use. Rocket parses and validates the form request, creates the structure, and calls your function. Bad form request? Rocket doesn’t call your function! What if you want to know if the form was bad? Simple! Change the type of... -
15
v86
x86 PC emulator and x86-to-wasm JIT, running in the browser
v86 is an open-source x86 PC emulator that runs entirely in the browser using WebAssembly for near-native performance. It translates x86 machine code into WebAssembly at runtime, enabling users to boot and run full operating systems without installing anything locally. The emulator supports a wide range of legacy systems, including Linux, Windows 95/98/2000, FreeDOS, and various experimental OSes. -
16
Trunk
Build, bundle & ship your Rust WASM application to the web
Trunk is a modern web application bundler specifically designed for Rust and WebAssembly projects, providing a streamlined workflow for building, packaging, and deploying frontend applications. It operates around a minimal-configuration philosophy, using a single HTML entry point to orchestrate the bundling of WebAssembly modules, JavaScript snippets, stylesheets, and other assets. This approach simplifies the build process compared to traditional bundlers by reducing the need for complex... -
17
Stremio Core
Types, addon system, UI models, core logic
Stremio Core is the Rust engine that powers Stremio’s apps by centralizing all reusable logic behind discovery, catalogs, metadata, streams, add-ons, and user/library state. It exposes a clean set of modules—types, addon_transport, and state_types—so apps can talk to add-ons, model UI state, and react to events without duplicating code. The architecture is inspired by Elm: immutable state, message-driven updates, and explicit side-effects (“effects”) keep behavior predictable and testable.... -
18
ffsend
Easily and securely share files from the command line
Easily and securely share files and directories from the command line through a safe, private and encrypted link using a single simple command. Files are shared using the Send service and may be up to 1GB. Others are able to download these files with this tool, or through their web browser. All files are always encrypted on the client, and secrets are never shared with the remote host. An optional password may be specified, and a default file lifetime of 1 (up to 20) download or 24 hours is... -
19
MooseStack
The developer framework for building analytical backends
MooseStack is an opinionated starter stack that assembles a modern web application foundation—project structure, build tooling, and deployment scripts—so teams can get from “blank repo” to a working product quickly. It provides a coherent layout for server and client code, standardizes environment configuration, and includes scripts to run the app locally with the same conventions you’ll use in staging or production. The stack favors convention over configuration: common decisions around... -
20
MoonZoon
Rust Fullstack Framework
MoonZoon is a full-stack Rust web framework that enables developers to build reactive web applications using a unified Rust codebase for both frontend and backend logic. It is built around a reactive programming model where UI updates are automatically triggered by changes in application state, reducing the need for manual DOM manipulation or complex state management patterns. The framework leverages WebAssembly for the frontend, allowing Rust code to run directly in the browser while... -
21
Percy
Build frontend browser apps with Rust + WebAssembly
Percy is a frontend framework that enables developers to build web applications using Rust and WebAssembly, offering an alternative to JavaScript-based frameworks. It provides a virtual DOM implementation and a declarative syntax through macros that allow developers to define UI components in a way similar to JSX or HTML templating systems. Percy supports multiple rendering strategies, including client-side rendering, server-side rendering, and hybrid approaches, giving developers... -
22
GraphQL Juniper
GraphQL server library for Rust
GraphQL is a data query language developed by Facebook intended to serve mobile and web application frontends. Juniper makes it possible to write GraphQL servers in Rust that are type-safe and blazingly fast. We also try to make declaring and resolving GraphQL schemas as convenient as Rust will allow. Juniper does not include a web server - instead it provides building blocks to make integration with existing servers straightforward. It optionally provides a pre-built integration for the... -
23
Dominator
Zero-cost ultra-high-performance declarative DOM library using FRP
Dominator is a high-performance declarative DOM library for Rust that focuses on building web applications with minimal overhead by directly manipulating real DOM nodes instead of relying on a virtual DOM. It uses a functional reactive programming model based on signals, allowing UI components to automatically update in response to state changes in an efficient and predictable manner. The library is designed to be “zero-cost,” meaning that abstractions compile down to highly optimized code... -
24
macroquad
Cross-platform game engine in Rust
macroquad is a lightweight, easy-to-use game library for the Rust programming language, offering developers a relatively simple and streamlined path to build games across platforms. The library is heavily inspired by simpler game frameworks (like raylib), but brings the ergonomic and safety features of Rust to 2D — and some minimal 3D — development. With Macroquad, you don’t need to worry about low-level graphics plumbing: it handles window and input management, rendering, UI, and more,... -
25
Russh
Rust SSH client & server library
Russh provides a Rust library for implementing SSH clients and servers with a modern, async-friendly design. It exposes building blocks for authentication, channel management, port forwarding, and key handling, allowing you to embed SSH functionality directly into Rust applications. The API is designed to be explicit and composable, making it possible to implement custom behaviors like reverse tunnels, interactive shells, and service multiplexing. Because performance and safety are central,...
