Search Results for "proxy list parser"

Scrapling is an adaptive web scraping framework designed to handle everything from a single HTTP request to large-scale, concurrent crawls. Built for modern websites, it intelligently adapts to structural changes by automatically relocating elements when page layouts update. The framework includes advanced fetchers capable of bypassing anti-bot protections such as Cloudflare Turnstile using stealth and browser automation techniques. Its powerful spider system supports multi-session crawling,...

AutoCrawler is a Python-based image crawling tool designed to automatically download large numbers of images from search engines using automated browser interaction. It uses Selenium and a Chrome browser driver to navigate image search pages and collect image sources based on keywords provided by the user. AutoCrawler supports multiprocess and multithreaded downloading, which allows it to retrieve images faster by running several tasks simultaneously. Users provide search terms through a...

PSpider is a lightweight web crawling framework written in Python designed to simplify the development of custom web spiders. It focuses on providing an easy-to-understand architecture while still supporting concurrent crawling for improved performance. It uses a multithreaded model that separates the crawling workflow into several components responsible for fetching, parsing, and saving data. Tasks are managed through queues, allowing different parts of the crawler to process work...

Lightweigh and easy to use tool to extract multiple proxies from a list of websites. IP Proxy Scraper is also available for windows, check it out here: https://sourceforge.net/projects/ipproxyscraper/

Spondulas is browser emulator and parser designed to retrieve web pages for hunting malware. It supports generation of browser user agents, GET/POST requests, and SOCKS5 proxy. It can be used to parse HTML files sent via e-mail. Monitor mode allows a website to be monitored at intervals to discover changes in DNS or content over time. Autolog mode creates an investigation file that documents redirection chains.

Proxy Tester is a proxy list tester and then it generates a wpad.dat file you can use for your browser based on working servers. wpad.dat file selects a random proxy server from the list of provided working proxy servers with each connection request.

ProxyCat supplies a whole solution of Proxy, including proxy server exploration, validation, scoring and proxy list exchanging by means of peer-to-peer. User can surf the internet via the auto-generated fast and stable proxy. HTTP/HTTPS SOCKS are both

This has a Python ICAP (Internet Content Adaptation Protocol) server and IRML parser. Using this, a web proxy can do rule-based adaptation of content before delivery to clients. Has a Python proxylet API and squid-icap-client code too.