Search Results for "web scraping"

Wombat is a lightweight web crawling and scraping library written in Ruby that focuses on extracting structured data from web pages using a concise domain-specific language (DSL). It is designed to simplify the process of defining how information should be collected from HTML documents without requiring large amounts of scraping boilerplate code. Developers can declare the data fields they want and specify selectors or rules for retrieving them, allowing Wombat to parse and return structured results. ...

Kimurai is an open source web scraping framework written in Ruby that simplifies the process of building automated data extraction tools. It provides a clean domain-specific language that allows developers to define scraping logic and data schemas with minimal boilerplate code. Kimurai can use AI-assisted extraction to identify where data resides in HTML pages, automatically generating selectors that are cached for future use so subsequent scraping runs operate with pure Ruby performance. ...

...By communicating directly with Chromium-based browsers through the DevTools protocol, Cuprite enables faster and more reliable browser automation for testing and scraping tasks. The driver integrates seamlessly with Capybara, allowing developers to write feature tests that simulate real user interactions with web applications. Because it uses headless Chrome by default, Cuprite is well suited for automated test environments and continuous integration pipelines. Developers can also run tests with a visible browser window for debugging purposes during development.

...Because of this low-level access, Ferrum offers greater flexibility and performance compared to traditional WebDriver-based automation tools. It is commonly used for tasks such as web scraping, automated testing, crawling, and screenshot or PDF generation.

Huginn is an open-source system for building agents that perform automated tasks by monitoring websites, APIs, emails, and more. Inspired by IFTTT, Huginn lets users create complex workflows and conditional logic to react to events and manage data. It’s self-hosted, highly customizable, and suitable for developers who want full control over automation without relying on third-party platforms.