Search Results for "linux proxy scraper" - Page 5
Sort By:
Showing 1379 open source projects for "linux proxy scraper"
View related business solutions-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
1
FastTunnel
Expose a local server to the internet
FastTunnel is a high-performance cross-platform intranet penetration tool. With it, you can expose intranet services to the public network for yourself or anyone to access. Unlike other penetration tools, the FastTunnel project is committed to creating an easy-to-extensible and easy-to-maintain intranet penetration framework. You can build your own penetration application by referencing the nuget package of FastTunnel.Core, and target the business extension functions you need. -
2
Ligolo-ng
An advanced, yet simple, tunneling/pivoting tool
Ligolo-ng is a simple, lightweight and fast tool that allows pentesters to establish tunnels from a reverse TCP/TLS connection using a tun interface (without the need of SOCKS). When running the relay/proxy server, a tun interface is used, packets sent to this interface are translated and then transmitted to the agent's remote network. You need to download the Wintun driver (used by WireGuard) and place the wintun.dll in the same folder as Ligolo. You can listen to ports on the agent and... -
3
X4G
Fast and modern gateway for VLESS tunneling on WebSocket and XHTTP
X4G is a self-hosted gateway for operating VLESS tunnels over WebSocket and two XHTTP transport modes. It also includes an internal HTTP proxy for clients that need a conventional proxy endpoint. A web dashboard displays service statistics, hourly traffic charts, live connections, activity records, and error logs. Administrators can create unlimited connection links with individual traffic quotas, bandwidth limits, expiration dates, ports, and simultaneous IP restrictions. Subscription... -
4
GO Simple Tunnel
A simple tunnel written in golang
GOST, or GO Simple Tunnel, is a flexible tunneling and proxy tool written in Go for building secure traffic forwarding chains. It supports proxying, port forwarding, reverse proxying, and tunnel-based access to services behind NAT or firewalls. The project is designed around protocol composition, allowing users to combine multiple protocols into multi-level forwarding chains. It can handle TCP and UDP forwarding, transparent proxying, DNS proxying, TUN/TAP workflows, and tun2socks scenarios.... -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
5
Wiretap
Wiretap is a transparent, VPN-like proxy server that tunnels traffic
Wiretap is a transparent, VPN-like proxy server that tunnels traffic through WireGuard. It is designed to proxy traffic without requiring special privileges to run, which makes it different from many traditional VPN tools that depend on privileged network interface setup. The project focuses on transparent routing, allowing client traffic to move through WireGuard-based paths toward the intended destination. Its documentation describes an end-to-end encryption model where packets are... -
6
Stowaway
Multi-hop Proxy Tool for pentesters
Stowaway is a Go-based multi-hop proxy tool created for authorized penetration testing, red-team labs, and security training environments. It uses an admin-and-agent model that allows operators to build a chain of connected nodes across segmented networks. The tool supports active and passive connection modes, making it adaptable to different network layouts. It can route traffic through SOCKS5 and HTTP proxy settings, and it also supports TCP, HTTP-style, and WebSocket-style upstream or... -
7
Proxyee
HTTP proxy server,support HTTPS & websocket
Proxyee is a JAVA-written HTTP proxy server library that supports HTTP, HTTPS, and WebSocket protocols, and supports MITM (Man-in-the-middle), which can capture and tamper with HTTP, and HTTPS packets. The CA certificate (src/resources/ca.crt) from the project needs to be imported to a trusted root certificate authority. The CA certificate (src/resources/ca.crt) from the project needs to be imported to a trusted root certificate authority. You can use the CertDownIntercept interceptor to... -
8
Surge-rules
The Surge rule set (DOMAIN-SET and RULE-SET) for the Surge for iOS
Surge-rules is a project devoted to maintaining and distributing rule sets compatible with the Surge proxy application (for macOS/iOS), intended for domain- or routing-based traffic management. Instead of manually building or maintaining Surge configuration files, users can rely on this repo to obtain up-to-date “RULE-SET” or “DOMAIN-SET” configurations that reflect community-curated lists of domains/IPs for direct access, proxying, blocking, or custom routing. Because network and domain... -
9
Sing-box
Sing-box multi-protocol proxy tool
Sing-box is a shell-script-based deployment project for setting up multi-protocol sing-box proxy environments across different hosting platforms and VPS scenarios. It is built around quick, automated installation rather than manual configuration, making it useful for users who want a bundled setup process. The project supports several proxy protocols, including VLESS Reality, VMess over WebSocket and TLS, Hysteria2, and Tuic. It also includes optional Nezha monitoring integration, which can... -
Build Data Resilience - Take the Assessment TodayIs your recovery strategy as strong as you think? Take this quick self-assessment to check your recovery readiness and gain tailored insights. In only 2 minutes, you'll learn where you fall on the recovery readiness scale.
-
10
InterceptSuite
A TLS MITM proxy for Non-HTTP traffic, with support for TLS upgrades
InterceptSuite is a cross‑platform, SOCKS5‑based MITM proxy specially designed to intercept, inspect, analyze, and manipulate encrypted network traffic at the TCP/TLS layer. It goes beyond HTTP‑focused tools like Burp Suite and ZAP by providing universal TLS interception—including STARTTLS and non‑HTTP protocols—offering deep visibility and control for security testing and debugging. InterceptSuite bridges this gap by providing a universal TLS interception engine that works with any... -
11
Toxiproxy
A TCP proxy to simulate network and system conditions for chaos
Toxiproxy is a framework for simulating network conditions. It's made specifically to work in testing, CI and development environments, supporting deterministic tampering with connections, but with support for randomized chaos and customization. Toxiproxy is the tool you need to prove with tests that your application doesn't have single points of failure. We've been successfully using it in all development and test environments at Shopify since October, 2014. See our blog post on resiliency... -
12
fanqiang
Over the Wall - Science Online
Over the wall-Scientific Internet, over-the-wall tool, over-the-wall tutorial project library. -
13
rtk
CLI proxy that reduces LLM token consumption
rtk is an open-source command-line proxy designed to optimize interactions between AI coding agents and the terminal by reducing unnecessary token consumption. When AI assistants execute shell commands during software development tasks, the resulting terminal output often contains large amounts of repetitive or irrelevant information that can overwhelm the model’s context window. RTK intercepts these command outputs and compresses them into concise summaries before sending them to the... -
14
Crawl4AI
Open-source LLM Friendly Web Crawler & Scraper
Crawl4AI is a high-performance, AI‑ready web crawler tailored for LLM data ingestion and RAG pipelines. It supports adaptive crawling heuristics (stopping when enough info is gathered), structured markdown output, and high-speed parallel execution. Designed to operate at scale with optional Docker deployment and framework integrations. -
15
go-mitmproxy
mitmproxy implemented with golang
go-mitmproxy is a Golang implementation of mitmproxy that supports man-in-the-middle attacks and parsing, monitoring, and tampering with HTTP/HTTPS traffic. Parses HTTP/HTTPS traffic and displays traffic details via a web interface. Supports a plugin mechanism for easily extending functionality. Various event hooks can be found in the examples directory. HTTPS certificate handling is compatible with mitmproxy and stored in the ~/.mitmproxy folder. If the root certificate is already trusted... -
16
MediaCrawler
Self-media platform crawler
MediaCrawler is an open-source tool for collecting public content from major Chinese social media platforms. It supports Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, Zhihu, and other services. The project uses Playwright to automate browser login and preserve authenticated sessions. It obtains required request signatures from the active browser context, avoiding complex JavaScript reverse engineering. Users can search by keyword, crawl specific posts, collect nested comments, and... -
17
V2Ray
Shell script that aims to make installing and managing V2Ray
This repository offers an interactive, one-command script that installs and manages a V2Ray-based proxy service on common Linux distributions. It automates routine but error-prone steps such as fetching the latest binaries, generating credentials (for example, UUIDs), writing JSON config files, opening firewall ports, and setting up a systemd service for automatic start. The script includes a menu-driven manager to modify ports, protocols, and users, restart the service, or view logs without hand-editing configuration files. ... -
18
XHS-Downloader
GUI/CLI tool for downloading Xiaohongshu
XHS-Downloader is a GUI/CLI tool for downloading Xiaohongshu (Little Red Book) content without watermarks, supporting both graphics and video posts. Prebuilt packages for Windows and macOS are available from Releases and GitHub Actions artifacts, so most users can run it by unzipping and launching the included executable. The project offers two execution paths—run the compiled app or run from source—and documents default download and configuration paths to simplify first use. Recent releases... -
19
PlexGuide.com
Rapidly deploy multiple-hasty Docker containers
Rapidly deploy multiple-hasty Docker Containers through Ansible with local or Unlimited Google HD Space! Project Statement: PlexGuide is an all-in-one media solution that deploys a Media Server through the use of your Local HD or Google Drive; serving as unlimited back-end storage. PGBlitz utilizes Ansible and Docker to streamline your Media Server while deploying multiple tools for your Server Operations. Deploys multiple programs/apps and functional within 10 - 30 seconds. Deploy PlexGuide... -
20
ScrapeGraphAI
Python scraper based on AI
Extracting content from websites and local documents using LLM. ScrapeGraphAI is a web scraping python library that uses LLM and direct graph logic to create scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown, etc.). Just say which information you want to extract and the library will do it for you. -
21
Verdaccio
A lightweight Node.js private proxy registry
A lightweight Node.js private proxy registry. Verdaccio is a simple, zero-config-required local private NPM registry. No need for an entire database just to get started. Verdaccio comes out of the box with its own tiny database, and the ability to proxy other registries (eg. npmjs.org), also introduces caching the downloaded modules along the way. For those who are looking to extend their storage capabilities, Verdaccio supports various community-made plugins to hook into services such as... -
22
Athens
A Go module datastore and proxy
Athens provides a server for Go Modules that you can run. It serves public code and your private code for you, so you don’t have to pull directly from a version control system (VCS) like GitHub or GitLab. There are many reasons why you’d want a proxy server such as security and performance. Athens is a project building on top of vgo (or go1.11+) trying to bring dependencies closer to you so you can count on repeatable builds even at a time when VCS is down. The big goal of Athens is to... -
23
Xray Core
Xray, Penetrates Everything. Also the best v2ray-core
Xray-core is an enhanced superset of v2ray-core, providing a high-performance privacy proxy framework supporting XTLS. It offers full compatibility with v2ray configs, supports multiple protocols, advanced routing, and is distributed as a single executable. -
24
Valtio
Valtio makes proxy-state simple for React and Vanilla
Valtio is a lightweight, proxy-based state management library for JavaScript and TypeScript, designed to simplify reactive state handling in both React and vanilla environments. It allows you to manage mutable state through normal JavaScript objects, while providing React hooks (useSnapshot) and subscription utilities to create optimized component re-renders and state observation. It uses ECMAScript Proxies and supports deep state tracking, making state mutation intuitive and powerful... -
25
goclone
Fast CLI tool for cloning entire websites for local browsing offline
goclone is a command-line utility designed to download and mirror complete websites to a local directory for offline access. It retrieves HTML pages, stylesheets, JavaScript files, images, and other assets from a target site and stores them on the user’s computer. It preserves the original site’s structure by maintaining relative links between pages, allowing the mirrored copy to function similarly to the live version when opened locally. Once a site has been cloned, users can browse the...
