Search Results for "scraping"

Happy DOM is a JavaScript implementation of a web browser without its graphical user interface. It includes many web standards from WHATWG DOM and HTML. The goal of Happy DOM is to emulate enough of a web browser to be useful for testing, scraping web sites, and server-side rendering. Happy DOM focuses heavily on performance and can be used as an alternative to JSDOM. Happy DOM now supports Declarative Shadow DOM which can be used for server-side rendering of web components. This package makes it possible to use Happy DOM with Jest.

Web data extraction (web data mining, web scraping) tool. It leverages well proved XML and text processing techologies in order to easely extract useful data from arbitrary web pages.

unfluff is a Node.js library designed to automatically extract the main content from an HTML document — stripping away navigation bars, ads, footers and other boilerplate to leave you with the “body content”, metadata (title, author, date) and other useful fields. It’s a tool very much aimed at content-analysis, web scraping, building datasets, or repurposing article text for downstream processing (like machine-learning or summarization). The API is simple: you feed in raw HTML and it returns a structured object with the extracted text and other fields. It supports caching internal representations to speed up repeated extractions. While its language support is best for English, it is still widely used in web-content-processing pipelines. ...

Simple-Scrape is a simple web-scraping library that allows for programmatic access to HTML code. No further techniques are needed and the library is very compact and thus easy to use.

SQLDOM is an easy and robust way to parse HTML directly into SQL tables, manipulate DOM nodes in a JQuery-like manner, and to render HTML from the SQL-based DOM. SQLDOM is written entirely in native T-SQL, and uses only temporary database objects (tempdb). No changes to user databases are required.

PHP web API designed to simplify object handling(loading, saving, querying, displaying, and editing), abstract the data from its display structure, and layout and allow the target data to be delivered to any supported format without special logic.

A robust website scraping framework that uses XML, XPath, RegEx and scripting to consume, parse, normalize and traverse HTML based on a set of seed URLs. Scrape.NET is built using C#, TidyForNet (the p-invoke only version) and HTML Tidy.

Xidel is a command line tool to download web pages and extract data from them. This data can be extracted using XPath/XQuery 3.0 (with a compatibility modes for XPath 2.0 and XQuery 1.0), JSONiq, CSS 3 selectors, and custom, pattern-matching templates that are like an annotated version of the processed page. It can download files over HTTP/S connections, follow redirections, links, or extracted values, and also process local files. The extracted values can then be exported as...