parsel

Stop writing brittle tests with hard-coded CSS, XPATH selectors and waits. Write tests that are meaningful to people, effortless to maintain, and reusable. The reason for this is because writing your tests using hard-coded CSS and XPATH selectors is like pouring concrete onto a specific UI implementation. This creates very high-maintenance tests that break at the slightest changes to the UI and are very hard for humans to read. UI-licious uses dynamic code analysis to understand the structure of your website based on the use of semantic HTML and ARIA accessibility attributes, as well as context of the previous commands, to identify which is the intended element to target for every command. This means that even if the HTML code for the UI changes underneath the code, the test is valid as long as the user journey remains the same. That said, your website doesn't to be written perfectly for UI-licious to work.

Scrapy is a fast high-level web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing. Built-in support for selecting and extracting data from HTML/XML sources using extended CSS selectors and XPath expressions, with helper methods to extract using regular expressions. Built-in support for generating feed exports in multiple formats (JSON, CSV, XML) and storing them in multiple backends (FTP, S3, local filesystem). Robust encoding support and auto-detection, for dealing with foreign, non-standard and broken encoding declarations.