Search Results for "data extraction"
Sort By:
Showing 30 open source projects for "data extraction"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
1
Firecrawl
Turn entire websites into LLM-ready markdown or structured data
Crawl and convert any website into LLM-ready markdown or structured data. Built by Mendable.ai and the Firecrawl community. Includes powerful scraping, crawling, and data extraction capabilities. Firecrawl is an API service that takes a URL, crawls it, and converts it into clean markdown or structured data. We crawl all accessible subpages and give you clean data for each. No sitemap is required. -
2
OCRBase
MD/.JSON Document OCR and structured data extraction API
OCRBase is a self-hostable document OCR and structured extraction system built to turn PDFs into machine-usable outputs at scale, aiming to bridge the gap between raw text extraction and production-ready pipelines. Instead of treating OCR as a one-off script, it presents an API-driven workflow where documents are submitted as jobs and processed through a queue-based architecture that can handle high throughput. The core output is designed for downstream automation, producing structured... -
3
DocStrange
Extract and convert data from any document, images, pdfs, word doc
DocStrange is an open-source document understanding and extraction library designed to convert complex files into structured, LLM-ready outputs such as Markdown, JSON, CSV, and HTML. Developed by Nanonets, the project combines OCR, layout detection, table understanding, and structured extraction into one end-to-end pipeline, which reduces the need to stitch together multiple separate services. It is built for developers who need high-quality parsing from scans, photos, PDFs, office files,... -
4
X-Crawl
Flexible Node.js AI-assisted crawler library
A high-performance web crawling and scraping framework for Node.js, designed for large-scale data extraction. -
$300 Free Credits to Build on Google CloudStart your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
-
5
watercrawl
AI-ready web crawler that extracts and structures website content
...WaterCrawl supports customizable extraction rules so users can focus only on relevant elements while ignoring unnecessary page components. WaterCrawl also offers real-time monitoring capabilities, allowing users to track crawling progress, performance metrics, and errors during large data collection jobs. Developers can integrate the tool into applications through a REST API and multiple client SDKs, enabling automated data pipelines and AI data preparation workflows. -
6
LiteParse
A fast, helpful, and open-source document parser
...It also includes mechanisms for validation and error handling, ensuring that outputs conform to expected schemas and reducing the need for manual postprocessing. The library is particularly useful for tasks such as data extraction, document processing, and building pipelines that require structured outputs from natural language input. -
7
Magnitude
Vision AI browser agent for automation, testing, and extraction
...This approach allows the agent to generalize better across complex and modern websites, making it more robust than traditional selector-based automation tools. Browser Agent by Magnitude supports a wide range of capabilities including navigation, interaction, data extraction, and automated verification through built-in testing features. Developers can use it to automate repetitive web tasks, integrate services without APIs, or build advanced browser-based agents. It also provides flexible abstraction levels, allowing both high-level task execution and precise low-level control of actions like mouse movements and keyboard input. -
8
Mobile Next
Model Context Protocol Server for Mobile Automation and Scraping
...It abstracts away platform-specific complexities, allowing developers and AI agents to interact with mobile devices using a consistent set of commands regardless of operating system. The system supports real devices, emulators, and simulators, making it suitable for testing, automation, and data extraction workflows in diverse development setups. One of its key innovations is its hybrid interaction model, which combines structured accessibility data with fallback screenshot-based analysis to ensure reliable automation even in complex UI scenarios. It is built to integrate seamlessly with modern AI agents, enabling multi-step workflows such as automated testing, form filling, and user journey simulation. -
9
KaraKeep
A self-hostable bookmark-everything app
...Automatic fetching of link titles, descriptions, and images streamlines saving content without manual edits, while rule-based management lets users define customized workflows. With support for image OCR and structured data extraction, Karakeep functions as a flexible personal knowledge base for researchers, content creators, and heavy bookmarkers. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
10
Documind
Open-source platform for extracting structured data from documents
Documind is an advanced document processing tool that leverages AI to extract structured data from PDFs. It is built to handle PDF conversions, extract relevant information, and format results as specified by customizable schemas. -
11
Wigolo
The go-to web for your AI coding agent
Wigolo is a local-first web intelligence system built for AI agents and coding assistants. It exposes search, fetching, crawling, extraction, research, monitoring, and autonomous gathering through MCP, REST, command-line tools, and SDKs. Its search engine combines results from multiple adapters using rank fusion, local reranking, and explainable evidence scores. The fetch router can escalate from ordinary HTTP requests to browser automation when it encounters dynamic pages or anti-bot... -
12
nhentai
A library for interacting with the nhentai API
nhentai is a JavaScript and TypeScript library designed to interact with the nhentai API and retrieve doujinshi metadata and content information. It enables developers to programmatically access galleries, titles, tags, covers, and page URLs from the nhentai platform. The library supports both CommonJS and ES6 module imports, making it easy to integrate into different Node.js projects. Developers can use it to fetch specific doujin entries, explore associated metadata, and process gallery... -
13
CodeGraph
Pre-indexed code knowledge graph for Claude Code, Codex, Cursor
CodeGraph is a local-first code intelligence tool that gives AI coding agents a pre-indexed understanding of a repository. Instead of forcing agents to repeatedly scan files with grep, glob, and read commands, it builds a searchable knowledge graph of symbols, relationships, call graphs, and code structure. The project uses deterministic parsing rather than LLM summaries, which makes its indexed data more predictable and grounded in the actual source tree. It is designed for tools such as... -
14
GenAIScript
Automatable GenAI Scripting
JavaScript-ish environment with convenient tooling for file ingestion, prompt development, and structured data extraction. A Microsoft tool that generates AI-powered text based on prompts, useful for content creation and automation. -
15
Gooo
Toolkit for developing web applications in Vue, Templ, and Go
...The project emphasizes simplicity and flexibility, enabling users to integrate its components into scripts or larger systems. While not as feature-heavy as enterprise frameworks, it serves as a foundation for experimentation and rapid prototyping in data extraction or automation tasks. Its design reflects a developer-centric approach, prioritizing extensibility and ease of modification over polished interfaces. -
16
HyperAgent
AI Browser Automation
...Built on top of Playwright, the framework allows developers to automate complex browser interactions using natural language commands rather than fragile selectors or hard-coded scripts. Instead of manually writing logic for clicking elements, extracting data, or navigating web pages, developers can instruct the agent in plain language and allow the AI layer to interpret and execute the task. This approach reduces the brittleness commonly associated with traditional automation scripts that break when the DOM structure changes. HyperAgent includes APIs such as page.ai() and page.extract() that allow structured data extraction and dynamic task execution through AI reasoning. -
17
LLM Scraper
Extract structured data from webpages using LLM-powered scraping
LLM Scraper is a TypeScript library designed to extract structured data from webpages using large language models. Instead of relying on fragile HTML selectors or manual parsing rules, the tool interprets webpage content with language models and converts it into structured data according to a defined schema. Developers can specify the data structure using tools such as Zod or JSON Schema, enabling the model to extract relevant information directly into typed objects. LLM Scraper integrates... -
18
HeadlessX
The undetected self-hosted browser automation platform
...The tool can perform tasks such as HTML extraction, screenshot generation, content parsing, and search result scraping while appearing like a normal user browser. Because it is self-hosted, organizations can run the platform on their own infrastructure to maintain privacy and control over automation workflows. -
19
nw_wrld
nw_wrld is an event-driven sequencer for triggering visuals
nw_wrld is a procedurally generated world-building engine tailored for game developers and interactive storytellers who want to craft rich, random yet coherent environments without hand-crafting every detail. It uses noise functions and modular terrain algorithms to generate expansive maps, diverse biomes, and layered features like rivers, mountain ranges, forests, and resource nodes. The system is designed to be extensible, letting developers plug in new generation rules or tweak parameters... -
20
Actors MCP Server
Model Context Protocol (MCP) Server for Apify's Actors
The Apify Actors MCP Server is a Model Context Protocol (MCP) server that enables AI assistants to interact with Apify Actors. This integration allows AI models to utilize various web scraping and automation tools provided by Apify, facilitating tasks such as data extraction and web automation. -
21
Browserbase MCP Server
Allow LLMs to control a browser with Browserbase and Stagehand
Browserbase MCP Server is a server implementation of the Model Context Protocol (MCP) that enables large language models to interact with web browsers programmatically through cloud-based automation. The project provides a standardized interface for connecting AI systems to real-world web environments, allowing them to navigate pages, extract structured data, and perform user-like actions such as clicking, typing, and form submission. It leverages Browserbase infrastructure along with... -
22
Claude Code Video Vision
Give Claude the ability to watch and understand videos
Claude Video Vision is a plugin designed for Claude Code that enables large language models to process and understand video content by transforming it into multimodal inputs the model can reason over. Instead of attempting to directly interpret raw video streams, the system extracts key frames using tools like ffmpeg and processes audio through transcription engines, converting both visual and auditory signals into structured inputs for the model. The result is a perception layer that feeds... -
23
Markdownify MCP Server
Convert files and web content into clean, usable Markdown easily
...It supports formats such as PDFs, images, audio with transcription, DOCX, XLSX, and PPTX, along with web sources like YouTube transcripts, Bing results, and general webpages. Markdownify MCP is designed to simplify content extraction and make data easier to read, share, and reuse in structured workflows. Developers can install dependencies, build, and run the server locally, then extend functionality by modifying its TypeScript-based tools and server logic. It also allows retrieval of existing Markdown files, making it useful for documentation, research, and AI-assisted workflows. ... -
24
BrowserNode
Make websites accessible for AI agents. Automate tasks online
Browsernode is an open-source TypeScript framework that allows AI agents to interact directly with web browsers in order to automate tasks and gather information from websites. The project acts as a bridge between AI models and browser automation tools, enabling language models to control web pages programmatically. Built as an implementation compatible with the Browser-use ecosystem, Browsernode allows agents to perform actions such as navigating pages, extracting information, filling... -
25
GPT Crawler
Crawl a site to generate knowledge files to create your own custom GPT
GPT Crawler is an open-source tool designed to automatically crawl websites and generate structured knowledge that can be used to build AI assistants and retrieval systems. It focuses on extracting high-quality textual content from web pages and preparing it in formats suitable for embedding, indexing, or fine-tuning workflows. The project is especially useful for teams that want to turn documentation sites or knowledge bases into conversational AI backends without building custom scrapers...