Search Results for "spider solitaire windows"

Colly provides a clean interface to write any kind of crawler/scraper/spider. With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving. Clean API. Fast (>1k request/sec on a single core) Manages request delays and maximum concurrency per domain. Automatic cookie and session handling. Sync/async/parallel scraping. Distributed scraping. Caching, automatic encoding of non-unicode...

GoSpider - Fast web spider written in Go. Fast web crawling. Brute force and parse sitemap.xml. Parse robots.txt. Generate and verify link from JavaScript files. Link Finder. Find AWS-S3 from response source. Find subdomains from the response source. Get URLs from Wayback Machine, Common Crawl, Virus Total, Alien Vault. Format output easy to Grep. Support Burp input. Crawl multiple sites in parallel.

Drop-in replacement for wkhtmltopdf built on Go, Electron and Docker. Athena is comprised of an Electron command line interface (CLI) tool, and a Go microservice for converting HTML to PDF documents. Athena transformed Arachne into a spider for challenging her as a weaver and/or weaving a tapestry that insulted the gods. Simple, Docker-powered PDF conversions. It was designed to do one thing and to do it well - PDF conversions; to work together with other programs; and to be able to handle...

DHT implements the bittorrent DHT protocol in Go. It contains two modes, the standard mode, and the crawling mode. The standard mode follows the BEPs, and you can use it as a standard dht server. The crawling mode aims to crawl as much metadata info as possible. It doesn't follow the standard BEPs protocol. With the crawling mode, you can build another BTDigg. The default crawl mode configuration costs about 300M RAM. Set MaxNodes and BlackListMaxSize to fit yourself. DHT aims to implement...

An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only. Spider gets a Request in Scheduler that has url to be crawled. Then Downloader downloads the result(html, json, jsonp, text) of the Request. The result is saved in Page for parsing in PageProcesser. Html parsing is based on goquery package. Json parsing is based on simple JSON package. Jsonp will...