Search Results for "extraction" - Page 2
Sort By:
Showing 775 open source projects for "extraction"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
1
watercrawl
AI-ready web crawler that extracts and structures website content
WaterCrawl is an open source web crawling and data extraction platform designed to transform website content into structured data suitable for machine learning and AI workflows. It enables developers and researchers to crawl web pages, extract meaningful information, and convert it into formats that are easier to process and analyze. It provides a modern crawling system that can automatically navigate links, control crawl depth, and collect content from targeted sections of a website. ... -
2
pyAudioAnalysis
Python Audio Analysis Library: Feature Extraction, Classification
...The project provides a collection of tools that allow developers to extract meaningful features from audio files and use those features for classification, segmentation, and analysis. The library supports multiple audio processing workflows, including feature extraction from raw audio signals, training of machine learning models, and automatic audio segmentation. It also includes utilities for visualizing audio features and analyzing patterns within sound recordings, which can be useful in applications such as speech recognition, music classification, and acoustic event detection. Because the library integrates machine learning algorithms with signal processing tools, it enables researchers to develop complete audio analysis pipelines using a single framework. -
3
AgentQL MCP
Model Context Protocol server that integrates AgentQL's data
The AgentQL MCP Server is a Model Context Protocol (MCP) server that integrates AgentQL's data extraction capabilities, enabling users to extract structured data from web pages using natural language prompts. -
4
dude uncomplicated data extraction
dude uncomplicated data extraction: A simple framework
Dude is a very simple framework for writing web scrapers using Python decorators. The design, inspired by Flask, was to easily build a web scraper in just a few lines of code. Dude has an easy-to-learn syntax. Dude is currently in Pre-Alpha. Please expect breaking changes. You can run your scraper from terminal/shell/command-line by supplying URLs, the output filename of your choice and the paths to your python scripts to dude scrape command. -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
5
JavaScript Obfuscator
A powerful obfuscator for JavaScript and Node.js
JavaScript Obfuscator is a Node.js library and CLI that transforms readable JavaScript into hardened, difficult-to-reverse code. It applies techniques such as identifier mangling, string array extraction/encoding, control-flow flattening, dead-code injection, and numeric literal transformations to disguise intent. Advanced options include self-defending code, domain locking, debug/console protection, and property key transformation, allowing you to tailor defenses to your threat model. The tool supports source maps and granular “threshold”/whitelist settings so you can balance protection with performance and debuggability. ... -
6
Scanopy
Clean network diagrams, One-time setup, zero upkeep
Scanopy is a powerful multi-modal data capture and analysis toolkit that enables users to collect, process, and visualize structured and unstructured information from a variety of sources in a flexible pipeline. It is built to handle complex scanning tasks — such as OCR, document analysis, audio transcription, network data capture, and image extraction — while providing unified APIs and workflows that make managing heterogeneous data sources seamless. Developers can compose custom pipelines that chain together transforms, filters, and exporters, enabling automation of tedious data preparation steps and accelerating insights with minimal code. The system places a premium on extensibility, allowing contributors to add new extractors or analysis modules tailored to specific industries or datasets. ... -
7
Symfony DomCrawler
Eases DOM navigation for HTML and XML documents
Symfony DomCrawler is a PHP component that provides powerful tools for navigating and extracting data from HTML and XML documents. It allows developers to parse, filter, and manipulate web pages using CSS selectors and XPath expressions. DomCrawler is widely used for web scraping, testing, and processing structured content, and integrates well with other Symfony components like BrowserKit. -
8
web-access
Skill for installing full networking capabilities for Claude Code
web-access is a tool designed to give AI agents structured and controlled access to web content, enabling them to retrieve, navigate, and process information from online sources in real time. It abstracts common web interactions such as page loading, data extraction, and navigation into reusable functions that can be invoked by agents. The system emphasizes safety and control, likely including mechanisms to manage permissions, rate limits, and content filtering. This allows agents to operate within defined boundaries while still benefiting from dynamic, up-to-date information. The architecture supports integration with broader agent frameworks, making it a key component for building systems that require external knowledge. ... -
9
Vectorize MCP Server
Official Vectorize MCP Server
The Vectorize MCP Server is a Model Context Protocol server that integrates with Vectorize, offering advanced vector retrieval and text extraction capabilities. -
Forever Free Full-Stack Observability | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
10
FModel
Unreal Engine Archives Explorer
FModel is a freeware application designed to explore Unreal Engine games archives, allowing users to delve into the assets and structures of games developed with Unreal Engine. -
11
BlockArrays.jl
BlockArrays for Julia
...The type BlockArray stores each block contiguously while the type PseudoBlockArray stores the full matrix contiguously. This means that BlockArray supports fast noncopying extraction and insertion of blocks while PseudoBlockArray supports fast access to the full matrix to use in for example a linear solver. -
12
Tamagui
Style React fast with 100% parity on React Native
...Tamagui also includes a full UI kit with both styled and unstyled components, enabling flexible design system creation. Its compiler performs advanced optimizations such as CSS extraction, tree flattening, and dead code elimination, reducing bundle size and improving rendering speed. The system includes robust theming capabilities with support for design tokens, responsive props, and dynamic themes like dark mode. -
13
MatImage
Image Processing library for Matlab
matImage is an open-source MATLAB library for image processing and analysis. It provides a variety of tools for image enhancement, segmentation, and feature extraction. It’s especially useful for users working on biomedical images or those needing detailed image analysis in MATLAB. -
14
kimuraframework
AI-first Ruby framework for building fast, flexible web scraping spide
Kimurai is an open source web scraping framework written in Ruby that simplifies the process of building automated data extraction tools. It provides a clean domain-specific language that allows developers to define scraping logic and data schemas with minimal boilerplate code. Kimurai can use AI-assisted extraction to identify where data resides in HTML pages, automatically generating selectors that are cached for future use so subsequent scraping runs operate with pure Ruby performance. ... -
15
LiteParse
A fast, helpful, and open-source document parser
...It also includes mechanisms for validation and error handling, ensuring that outputs conform to expected schemas and reducing the need for manual postprocessing. The library is particularly useful for tasks such as data extraction, document processing, and building pipelines that require structured outputs from natural language input. -
16
video2robot
End-to-end pipeline converting generative videos
video2robot is an end-to-end open-source pipeline that converts generative video or prompt-driven motion content into executable humanoid robot motion sequences, enabling researchers and developers to go from high-level action descriptions or videos to robot-ready motion data. The pipeline supports both prompt-to-video generation using models like Veo/Sora and video upload processing, followed by human pose extraction through a 3D pose model and retargeting of that motion to robot joints using a general motion retargeting system. This workflow allows users to generate robot motion files that specify joint angles, root positions, and orientations that can be deployed on supported robot platforms (e.g., Unitree models). Video2robot includes scripts for each stage of the pipeline (generation, extraction, conversion, visualization) and can run as a CLI or through a basic web UI. -
17
Trafilatura
Python & command-line tool to gather text on the Web
Trafilatura is a Python package and command-line tool designed to gather text on the Web. It includes discovery, extraction and text-processing components. Its main applications are web crawling, downloads, scraping, and extraction of main texts, metadata and comments. It aims at staying handy and modular: no database is required, the output can be converted to various commonly used formats. Going from raw HTML to essential parts can alleviate many problems related to text quality, first by avoiding the noise caused by recurring elements (headers, footers, links/blogroll etc.) and second by including information such as author and date in order to make sense of the data. ... -
18
Ferret
Declarative web scraping
A web scraping system aiming to simplify data extraction from the web. ferret has a declarative query language that makes it easy to focus on the data that you need to get. ferret has the ability to scrape JS rendered pages, handle all page events, and emulate user interactions. the ferret was designed as a library from the ground up. it can be easily embedded into any Go application. ferret helps you to focus on the data you need using an easy-to-learn declarative language. ferret uses Chrome/Chromium via Chrome Devtools Protocol to handle dynamically rendered web pages. ferret is extremely extensible, and creating custom functions and types is super easy. ferret allows users to focus on the data. ... -
19
Bespoke Curator
Synthetic data curation for post-training and data extraction
...It supports workflows where models are used to produce synthetic examples that can later be refined into reliable training datasets for reasoning, question answering, or structured information extraction tasks. Curator includes tools for monitoring data generation processes and managing dataset quality while large batches of examples are being created. The framework also integrates with multiple inference systems and APIs, allowing users to generate data using different model providers or open-source inference engines. -
20
The Web MCP
A powerful Model Context Protocol (MCP) server
Bright Data’s Web MCP server gives AI assistants robust, real-time web capabilities through an MCP interface designed to avoid blocks, rate limits, and CAPTCHAs. It presents search, crawl, navigate, and extraction tools that agents can call directly, replacing brittle scraping prompts with typed operations. The README markets it as a “gateway” to the live web so assistants don’t fall back to stale training data. Bright Data also advertises a getting-started tier with a free monthly allotment, plus options for remote or self-hosted operation depending on governance needs. ... -
21
warp
A super-easy, composable, web server framework for warp speeds
The fundamental building block of warp is the Filter, they can be combined and composed to express rich requirements on requests. A Filter in warp is essentially a function that can operate on some input, either something from a request, or something from a previous Filter, and returns some output, which could be some app-specific type you wish to pass around, or can be some reply to send back as an HTTP response. That might sound simple, but the exciting part is the combinators that exist... -
22
Hacks
A collection of hacks and one-off scripts
...Rather than being a single cohesive application, it serves as a repository of practical command-line tools that can be used independently or combined into workflows. The scripts cover a wide range of tasks, including URL manipulation, parameter replacement, data extraction, and reconnaissance automation. Many of the tools in the repository are designed for efficiency and simplicity, enabling users to perform complex operations with minimal overhead. It is particularly popular among security researchers and developers who need quick, flexible solutions for niche problems. -
23
GoWall
A tool to convert a Wallpaper's color scheme / palette, image to pixel
Gowall is a versatile command-line tool for processing images, initially created to convert wallpapers to match specific color schemes. It has evolved to include features like image-to-pixel-art conversion, color palette extraction, background removal, and more, making it a powerful utility for image manipulation. -
24
LangChain Extract
Did you say you like data?
LangChain Extract is an open-source reference application designed to demonstrate how large language models can be used to extract structured data from unstructured text and document files. The project implements a lightweight web service that allows developers to define extraction schemas and apply them to various sources such as plain text, HTML, or PDF documents. Built using FastAPI and the LangChain framework, the application exposes a REST API that can process documents and return structured outputs that match user-defined JSON schemas. Developers can create reusable “extractors” that define what type of information should be pulled from a document, along with example prompts that improve extraction quality through in-context learning. -
25
PDFMathTranslate
PDF scientific paper translation with preserved formats
PDFMathTranslate is a Python-based tool that uses AI translation to convert academic PDFs into bilingual (e.g. Chinese-English) documents while preserving formatting, including math notation. It supports OCR-enhanced content and offers CLI, GUI, Docker, and Zotero integration under AGPL v3.
