Search Results for "python web crawler"

The main function of the crawler agent IP pool project is to regularly collect free agents published on the Internet for verification and storage, and to regularly verify and store agents to ensure the availability of agents, and to provide API and CLI. At the same time, you can also expand the proxy source to increase the quality and quantity of the proxy pool IP.

XX-Net is an easy-to-use, anti-censorship web proxy tool from China. It includes GAE_proxy and X-Tunnel, with support for multiple platforms.

mitmproxy is an open source, interactive SSL/TLS-capable intercepting HTTP proxy, with a console interface fit for HTTP/1, HTTP/2, and WebSockets. It's the ideal tool for penetration testers and software developers, able to debug, test, and make privacy measurements. It can intercept, inspect, modify and replay web traffic, and can even prettify and decode a variety of message types. Its web-based interface mitmweb gives you a similar experience as Chrome's DevTools, with the addition of...

Jupyter Server Proxy lets you run arbitrary external processes (such as RStudio, Shiny Server, Syncthing, PostgreSQL, Code Server, etc) alongside your notebook server and provide authenticated web access to them using a path like /rstudio next to others like /lab. Alongside the Python package that provides the main functionality, the JupyterLab extension (@jupyterhub/jupyter-server-proxy) provides buttons in the JupyterLab launcher window to get to RStudio for example.

HTTP Replicator is a general purpose caching proxy server written in python. It reduces bandwidth by merging concurrent downloads and building a local 'replicated' file hierarchy, similar to wget -r. The cache is also accessible through a web interface

Open Source Software Tool for installing and configuring Squid: Proxy Server and Web Cache Daemon. Developed By: Ashwin Tumma

Proxy server that performs proof of concept session hijacking by receiving cookies from one or more NomNom Agents. It then offers the user an option to switch to any intercepted session. More features and site specific options later.

Shweby is a new ICAP enabled proxy server. This proxy passes HTTP requests and responses through one or many ICAP servers in order to be modified in their way to the origin server or to the client.

This project is a Python-based HTTP web proxy server that hooks into MySQL to store a full history of your browsing. Allows you to check out statistics about your browsing habits. Creates a personal portal page, has search features, multi-user, filters.