Search Results for "scraping"

The UI Vision RPA software is the tool for visual process automation, codeless UI test automation, web scraping and screen scraping. Automate tasks on Windows, Mac and Linux. The UI Vision RPA core is open-source with enterprise security. The free and open-source browser extension can be extended with local apps for desktop UI automation. UI.Vision RPA's computer-vision visual UI testing commands allow you to write automated visual tests with UI.Vision RPA - this makes UI.Vision RPA the first and only Chrome and Firefox extension (and Selenium IDE) that has "👁👁 eyes". ...

...It fills a monitoring niche by being the glue between applications that do not export their own internal state (other than via logs) and existing monitoring systems, such that system operators do not need to patch those applications to instrument them or writing custom extraction code for every such application. The extraction is controlled by mtail programs which define patterns and actions. Metrics are exported for scraping by a collector as JSON or Prometheus format over HTTP, or can be periodically sent to a collectd, StatsD, or Graphite collector socket. Precompiled binaries for released versions are available in the Releases page on Github. Using the latest production release binary is the recommended way of installing mtail.

...We created Browser Pool because we regularly needed to execute tasks concurrently in many headless browsers and their pages, but we did not want to worry about launching browsers, closing browsers, restarting them after crashes and so on. We also wanted to easily and reliably manage the whole browser/page lifecycle. You can use Browser Pool for scraping the internet at scale, testing your website in multiple browsers at the same time or launching web automation robots.

...Save screenshots for the crawling evidence, emulate devices and user agents, priority queue for crawling efficiency, obey robots.txt, and more. The static crawlers are based on simple requests to HTML files. They are generally fast, but fail scraping the contents when the HTML dynamically changes on browsers. Dynamic crawlers based on PhantomJS and Selenium work magically on such dynamic applications. However, PhantomJS's maintainer has stepped down and recommended to switch to Headless Chrome, which is fast and stable. This crawler is dynamic and based on Headless Chrome.

lua-resty-waf is a web application firewall implemented in Lua for OpenResty/NGINX, designed to run inline at the edge with low overhead. It inspects requests and responses during NGINX phases, applying rule logic and anomaly scoring to detect patterns like SQL injection, cross-site scripting, and protocol abuse. Rules are organized into policies with configurable actions—block, log, or allow—and can leverage shared dictionaries for counters, rate limits, and caching decisions. Because it...

...The project was initially just to help the author get to grips with python but as of version 1.0 the project is stable enough to be used regularly and has proven to be a very useful tool. pyShowRename interfaces directly with the free epguides.com website using HTML scraping and retains a local cache of available TV Shows. This can be manually updated when new shows become available. Changelog ------------------------------------------------------------------------------ 2012-01-01 :: Initial launch. 2012-12-19 :: Code overhaul. Moved superfluous functions and GUI controls. Finalised reversion feature so UNDO WORKS! ...

Aracnis is a Java based framework for building distributed web spiders. These spiders can be used to accomplish a variety of tasks, for example, screen-scraping and link integrity checking.