Search Results for "data extraction"
Sort By:
Showing 26 open source projects for "data extraction"
View related business solutions-
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
Auth0 B2B Essentials: SSO, MFA, and RBAC Built InAuth0's B2B Essentials plan gives you everything you need to ship secure multi-tenant apps. Unlimited orgs, enterprise SSO, RBAC, audit log streaming, and higher auth and API limits included. Add on M2M tokens, enterprise MFA, or additional SSO connections as you scale.
-
1
AgentQL MCP
Model Context Protocol server that integrates AgentQL's data
The AgentQL MCP Server is a Model Context Protocol (MCP) server that integrates AgentQL's data extraction capabilities, enabling users to extract structured data from web pages using natural language prompts. -
2
newpipeextractor
Library for extracting streaming site data without official APIs
...It handles many low-level tasks involved in web data extraction, including parsing responses, managing platform-specific logic, and handling errors, allowing developers to focus on implementing application features rather than scraping mechanics. Each supported service is implemented through its own extractor components that conform to a common interface, enabling consistent access to data across different platforms. -
3
Vectorize MCP Server
Official Vectorize MCP Server
The Vectorize MCP Server is a Model Context Protocol server that integrates with Vectorize, offering advanced vector retrieval and text extraction capabilities. -
4
Wiseflow
Enhance any agent's browser use skill
Wiseflow is an open-source information extraction and knowledge discovery system designed to collect, filter, and organize valuable information from large volumes of online content. The platform continuously monitors specified sources such as websites, social platforms, and other digital channels to identify relevant data according to user-defined interests or topics. By combining web crawling, content parsing, and large language model analysis, the system extracts concise insights from raw information streams and converts them into structured data that can be stored or analyzed. ... -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
5
Magnitude
Vision AI browser agent for automation, testing, and extraction
...This approach allows the agent to generalize better across complex and modern websites, making it more robust than traditional selector-based automation tools. Browser Agent by Magnitude supports a wide range of capabilities including navigation, interaction, data extraction, and automated verification through built-in testing features. Developers can use it to automate repetitive web tasks, integrate services without APIs, or build advanced browser-based agents. It also provides flexible abstraction levels, allowing both high-level task execution and precise low-level control of actions like mouse movements and keyboard input. -
6
npm-pdfreader
Parse text and tables from PDF files.
npm-pdfreader is a Node.js library for reading text and parsing tables from PDF files. It supports tabular data with automatic column detection and rule-based parsing, making it useful for extracting structured data from PDFs. -
7
The Web MCP
A powerful Model Context Protocol (MCP) server
Bright Data’s Web MCP server gives AI assistants robust, real-time web capabilities through an MCP interface designed to avoid blocks, rate limits, and CAPTCHAs. It presents search, crawl, navigate, and extraction tools that agents can call directly, replacing brittle scraping prompts with typed operations. The README markets it as a “gateway” to the live web so assistants don’t fall back to stale training data. Bright Data also advertises a getting-started tier with a free monthly allotment, plus options for remote or self-hosted operation depending on governance needs. ... -
8
spider_collection
Collection of Python web scraping scripts for data extraction tasks
spider_collection is a collection of Python web crawler scripts created primarily for experimentation, learning, and practical scraping tasks. spider_collection gathers multiple independent spiders designed to collect data from different platforms and services, demonstrating a variety of scraping techniques and workflows. These crawlers make use of common Python scraping tools such as requests, parsel, BeautifulSoup, and the Scrapy framework to extract structured information from web pages.... -
9
DotnetSpider
Lightweight .NET framework for fast web crawling and data scraping
DotnetSpider is a web crawling and data extraction framework built on the .NET Standard platform. It is designed to help developers create efficient and scalable crawlers for collecting structured data from websites. It provides a high-level API that simplifies the process of defining spiders, managing requests, and extracting content from web pages. Developers can create custom spiders by extending base classes and configuring pipelines that handle downloading, parsing, and storing collected data. ... -
Add Two Lines of Code. Get Full APM.Works out of the box for Rails, Django, Express, Phoenix, and more. Monitoring exceptions and performance in no time.
-
10
Article Extractor
To extract main article from given URL with Node.js
A Node.js library for extracting main content from web articles, removing unnecessary clutter like ads and navigation elements. -
11
Open Semantic Search
Open source semantic search and text analytics for large document sets
...It integrates text mining and analytics capabilities that allow users to examine relationships, topics, and structured data within document collections. -
12
WanGP
AI video generator optimized for low VRAM and older GPUs use
...Wan2GP provides a full web-based interface that simplifies interaction with complex generative pipelines, making it easier to configure prompts, models, and rendering settings. It also integrates a wide range of utilities such as prompt enhancement, mask editing, motion design, and extraction tools for pose, depth, and flow data to support advanced video workflows. -
13
MCiSEE
All of Minecraft, EASILY get Minecraft resources
MCiSEE is an open-source project designed to integrate Minecraft with computer vision and artificial intelligence experiments. The system focuses on capturing visual information from the game environment and exposing it to external programs for analysis or machine learning research. By converting gameplay data into visual or structured formats, MCiSEE enables researchers and developers to build AI agents capable of interacting with the Minecraft environment. The project can be used as a... -
14
DeepCamera
Open-Source AI Camera. Empower any camera/CCTV
...SharpAI yolov7_reid is an open-source Python application that leverages AI technologies to detect intruders with traditional surveillance cameras. The source code is here It leverages Yolov7 as a person detector, FastReID for person feature extraction, Milvus the local vector database for self-supervised learning to identify unseen persons, Labelstudio to host images locally and for further usage such as label data and train your own classifier. It also integrates with Home-Assistant to empower smart homes with AI technology. -
15
sharp
High performance Node.js image processing module
...Colour spaces, embedded ICC profiles and alpha transparency channels are all handled correctly. Lanczos resampling ensures quality is not sacrificed for speed. As well as image resizing, operations such as rotation, extraction, compositing and gamma correction are available. Most modern macOS, Windows and Linux systems running Node.js v10+ do not require any additional install or runtime dependencies. This module supports reading JPEG, PNG, WebP, AVIF, TIFF, GIF and SVG images. Output images can be in JPEG, PNG, WebP, AVIF and TIFF formats as well as uncompressed raw pixel data. ... -
16
Markdownify MCP Server
Convert files and web content into clean, usable Markdown easily
...It supports formats such as PDFs, images, audio with transcription, DOCX, XLSX, and PPTX, along with web sources like YouTube transcripts, Bing results, and general webpages. Markdownify MCP is designed to simplify content extraction and make data easier to read, share, and reuse in structured workflows. Developers can install dependencies, build, and run the server locally, then extend functionality by modifying its TypeScript-based tools and server logic. It also allows retrieval of existing Markdown files, making it useful for documentation, research, and AI-assisted workflows. ... -
17
browserable
Open source and self-hostable browser automation library for AI agents
Browserable is an open-source browser automation framework designed specifically for AI agents that need to interact with web interfaces in a human-like way. The project provides tools that allow automated agents to navigate websites, click buttons, fill out forms, and extract information from pages without manual scripting of each step. Built primarily in JavaScript, the framework offers both a developer-friendly SDK and a REST API that allow integration with AI applications and automation... -
18
Firecrawl MCP Server
Adds powerful web scraping and search to Cursor and Claude
firecrawl-mcp-server is the official MCP integration for Firecrawl that brings high-recall web scraping, crawling, and search into IDEs and agent runtimes. It exposes tools for single-page scrape, multi-URL batch jobs, site discovery, and search enrichment, returning cleaned, structured content suitable for downstream LLM reasoning. The server is designed to run with Firecrawl’s hosted API or self-hosted deployments, making it flexible for enterprise data-governance requirements. Built-in... -
19
MBR-Bulk-WP-Detector
A free WP plugin that lets you check unlimited URLs
MBR Bulk WP Detector is a free WordPress plugin that lets you check unlimited URLs right from your own dashboard. No subscriptions, no URL limits, and your data stays completely private on your server. What Can You Do With It? The basics are simple: Paste a list of URLs (or upload a CSV file), click a button, and boom—you’ve got a clear breakdown of which sites are running WordPress and which aren’t. But it gets better… Turn on Deep Scan mode, and you’ll also discover what... -
20
Exifr
The fastest and most versatile JS EXIF reading library
Exifr is a fast and very versatile JavaScript EXIF reading library that works everywhere, parses everything and handles just about anything you throw at it. It can handle any input: buffers, url, <img> tag and more; .jpg, .tif, and .heic files; and TIFF (EXIF, GPS, etc.), XMP, ICC, IPTC, JFIF segments. It skips parsing tags you don’t need, and reads only the first few bytes. There’s no need to read the whole file to see if there’s an EXIF file in it, or extract all the data when you just... -
21
Roomba
A Node.js tool to examine the correctness of Open Data Metadata
Linked Open Data (LOD) has emerged as one of the largest collection of interlinked datasets on the web. Benefiting from this mine of data requires the existence of descriptive information about each dataset in the accompanying metadata. Such meta information is currently very limited to few data portals where they are usually provided manually thus giving little or bad quality insights. To address this issue, we propose a scalable automatic approach for extracting, validating and generating... -
22
Pumilio is a web-based sound analysis and archive system for almost any kind of sound file with tools to see the spectrogram of the sound, select regions for further analysis and insertion in a database, filtering, and many other manipulations.
-
23
Framework for making Windows applications that are one .exe file in AutoHotKey_L,C++,C#, VB.NET,Java,Groovy,Common Lisp,Nemerle,Ruby,Python,PHP,Lua,Tcl,Perl,Jint,S#,WSH VBScript,HTML/JavaScript/CSS,COM, PowerShell without compiling . For .NET 4.
-
24
The Cornell Web Lab Collaboration Server is a suite of tools and services for GUI-based extraction, analysis and sharing of archived web data. See http://weblab.infosci.cornell.edu/ and http://www.cs.cornell.edu/~weigel for details about the project.
-
25
