Search Results for "data extract" - Page 2
Sort By:
Showing 399 open source projects for "data extract"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
1
JSLT
JSON query and transformation language
JSLT is a complete query and transformation language for JSON. The language design is inspired by jq, XPath, and XQuery. -
2
AI-Crawler
Crawl a website starting from a URL, find relevant pages
...Users can define their data requirements in plain English, and the system will interpret those instructions to crawl a domain and extract structured data. The tool supports output formats such as JSON and Markdown, and it can generate or accept schemas to ensure that extracted data is structured according to application needs. It is designed as a low-code solution, reducing the complexity of building and maintaining custom scraping pipelines. -
3
DouK-Downloader
TikTok releases/likes/compilations/live streams/videos/atlases/music
DouK-Downloader is a fully open-source data acquisition and media downloading tool designed to extract, collect, and download content from TikTok and its Chinese counterpart Douyin at scale. Built using Python and modern asynchronous networking libraries such as HTTPX, it enables batch downloading of videos, images, live streams, and metadata from accounts, playlists, and individual links. -
4
Spider
High-performance Rust web crawler and scraper for large-scale data
...These capabilities make the project suitable for building search indexers, data extraction pipelines, & SEO analysis tools. -
Build Data Resilience - Take the Assessment TodayIs your recovery strategy as strong as you think? Take this quick self-assessment to check your recovery readiness and gain tailored insights. In only 2 minutes, you'll learn where you fall on the recovery readiness scale.
-
5
PDFCraft
PDFCraft is a free, privacy-focused PDF toolkit
...At its core, the project provides a clean, modern UI where you can rearrange pages, annotate text, insert images, fill forms, and export to multiple formats, all without needing a heavyweight commercial PDF suite. But beyond manual editing, it also offers a programmable layer so developers can write scripts to batch process documents, generate templated reports, or extract structured data from PDFs for integration in workflows. The design emphasizes quality and compatibility: output PDFs render accurately across readers, preserve metadata, and support interactive elements like hyperlinks and form fields. -
6
watercrawl
AI-ready web crawler that extracts and structures website content
WaterCrawl is an open source web crawling and data extraction platform designed to transform website content into structured data suitable for machine learning and AI workflows. It enables developers and researchers to crawl web pages, extract meaningful information, and convert it into formats that are easier to process and analyze. It provides a modern crawling system that can automatically navigate links, control crawl depth, and collect content from targeted sections of a website. ... -
7
DiscordChatExporter
Saves Discord chat logs to a file
...Overall, DiscordChatExporter provides a reliable way to extract and preserve Discord communications outside the platform. -
8
go-i18n
Translate your Go program into multiple languages
go-i18n is a Go package and a command that helps you translate Go programs into multiple languages. Supports pluralized strings for all 200+ languages in the Unicode Common Locale Data Repository (CLDR). Code and tests are automatically generated from CLDR data. Supports strings with named variables using text/template syntax. Supports message files of any format (e.g. JSON, TOML, YAML). Use goi18n extract to extract all i18n.Message struct literals in Go source files to a message file for translation. Create an empty message file for the language that you want to add (e.g. translate.es.toml). ... -
9
cognee
Deterministic LLMs Outputs for AI Applications and AI Agents
...Any kind of data works; unstructured text or raw media files, PDFs, tables, presentations, JSON files, and so many more. Add small or large files, or many files at once. We map out a knowledge graph from all the facts and relationships we extract from your data. Then, we establish graph topology and connect related knowledge clusters, enabling the LLM to "understand" the data. -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
10
GoldenCheetah
Performance Software for Cyclists, Runners, Triathletes and Coaches
Analyze using summary metrics like BikeStress, TRIMP, or RPE. Extract insight via models like Critical Power and W'bal. Track and predict performance using models like Banister and PMC. Optimize aerodynamics using Virtual Elevation. Train indoors with ANT and BTLE trainers. Upload and Download with many cloud services including Strava, Withings, and Today's Plan. Import and export data to and from a wide range of bike computers and file formats. -
11
DataProfiler
Extract schema, statistics and entities from datasets
DataProfiler is an AI-powered tool for automatic data analysis and profiling, designed to detect patterns, anomalies, and schema inconsistencies in structured and unstructured datasets. The DataProfiler is a Python library designed to make data analysis, monitoring, and sensitive data detection easy. Loading Data with a single command, the library automatically formats & loads files into a DataFrame. Profiling the Data, the library identifies the schema, statistics, entities (PII / NPI), and... -
12
JSONPath Plus
A fork of JSONPath
Analyse, transform, and selectively extract data from JSON documents (and JavaScript objects). JSON path-plus expands on the original specification to add some additional operators and makes explicit some behaviors the original did not spell out. Try the browser demo or Runkit (Node). -
13
esquisse
RStudio add-in to make plots interactively with ggplot2
The purpose of this add-in is to let you explore your data quickly to extract the information they hold. You can create visualization with {ggplot2}, filter data with {dplyr} and retrieve generated code. This addin allows you to interactively explore your data by visualizing it with the ggplot2 package. It allows you to draw bar plots, curves, scatter plots, histograms, boxplot and sf objects, then export the graph or retrieve the code to reproduce the graph. ... -
14
Alternative JSON parser for Go
One of the fastest alternative JSON parser for Go
It does not require you to know the structure of the payload (eg. create structs), and allows accessing fields by providing the path to them. It is up to 10 times faster than standard encoding/json package (depending on payload size and usage), allocates no memory. Originally made for a project that relies on a lot of 3rd party APIs that can be unpredictable and complex. Encoding/json requires you to know exactly your data structures, or if you prefer to use map[string]interface{} instead,... -
15
jsoup
Java library for working with real-world HTML
jsoup is a Java library for working with real-world HTML. It provides a very convenient API for fetching URLs and extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors. jsoup implements the WHATWG HTML5 specification, and parses HTML to the same DOM as modern browsers do. jsoup is designed to deal with all varieties of HTML found in the wild; from pristine and validating, to invalid tag-soup; jsoup will create a sensible parse tree. The parser will make... -
16
Weibo Crawler
Python crawler for collecting and downloading Sina Weibo user data
weibo-crawler is a Python-based data collection tool designed to retrieve information from Sina Weibo user accounts. It automates the process of gathering posts, user profile details, and engagement metrics from one or more target accounts. weibo-crawler can extract comprehensive information about users, including profile attributes such as nickname, follower count, following count, and account metadata. -
17
docext
An on-premises, OCR-free unstructured data extraction
docext is a document intelligence toolkit that uses vision-language models to extract structured information from documents such as PDFs, forms, and scanned images. The system is designed to operate entirely on-premises, allowing organizations to process sensitive documents without relying on external cloud services. Unlike traditional document processing pipelines that rely heavily on optical character recognition, docext leverages multimodal AI models capable of understanding both visual... -
18
Chandra
OCR model for complex documents with layout-aware structured outputs
Chandra is an advanced OCR model designed to extract and structure information from complex documents such as tables, forms, handwritten notes, and mathematical content. It focuses on preserving full document layout, meaning that extracted text is accompanied by positional metadata like bounding boxes for each element. Chandra supports multiple output formats including Markdown, HTML, and JSON, making it suitable for downstream processing and integration into data pipelines. ... -
19
Jbuilder
Generate JSON objects with a Builder-style DSL
Jbuilder gives you a simple DSL for declaring JSON structures that beats manipulating giant hash structures. This is particularly helpful when the generation process is fraught with conditionals and loops. You can either use Jbuilder stand-alone or directly as an ActionView template language. When required in Rails, you can create views à la show.json.jbuilder (the json is already yielded). Fragment caching is supported, it uses Rails.cache and works like caching in HTML templates. If your... -
20
BettaFish
Public opinion analysis system
BettaFish is an open-source, multi-agent public opinion analysis system built to automate the collection, deep analysis, and reporting of social media data at scale through conversational queries. It uses a modular architecture of specialized agents that collaborate to crawl mainstream platforms, extract multimodal content like text and short video, and synthesize insights through both statistical and large language model techniques. With a design that lets users pose questions in natural language and receive structured reports, charts, and visualizations, the system aims to break information cocoons and provide comprehensive views of trends and public sentiment. ... -
21
monolith
CLI tool for saving complete web pages as a single HTML file
A data hoarder’s dream come true, bundle any web page into a single HTML file. You can finally replace that gazillion of open tabs with a gazillion of .html files stored somewhere on your precious little drive. Unlike the conventional “Save page as”, monolith not only saves the target document, it embeds CSS, image, and JavaScript assets all at once, producing a single HTML5 document that is a joy to store and share. If compared to saving websites with wget -mpk, this tool embeds all assets... -
22
Figma Sprite Generator
A Figma plugin to generate sprite sheets and JSON files from selected
Figma Sprite Generator is a Figma plugin designed to create sprite sheets and corresponding JSON metadata from selected icons. This tool is ideal for designers and developers working on web projects or UI libraries, as it automates sprite generation for more efficient workflows. Users can quickly convert their icons into PNG sprites and extract JSON files with icon dimensions and positions, streamlining the process of incorporating sprites into websites or applications. -
23
Extractous
Fast and efficient unstructured data extraction
Extractous is a Rust-based unstructured data extraction library focused on fast local parsing of documents and other content-heavy files. Its purpose is to extract text and metadata efficiently from formats such as PDF, Word, HTML, email archives, images, and more, without depending on external APIs or separate parsing servers. The project emphasizes performance and low memory usage, and its maintainers describe it as a local-first alternative to heavier extraction stacks. ... -
24
PDFPatcher
A versatile toolkit for PDF manipulation
PDFPatcher (aka “PDF补丁丁”) is a versatile toolkit for PDF manipulation—editing document metadata, bookmarks, page layout, content restrictions, rotation, compression, merging/splitting, image extraction, and more, all within an intuitive interface. Merge/split PDFs or images, preserve or add bookmarks, and set page dimensions. Batch style/color/target changes, regex/XPath search/replace, mid‑page positioning. Modify PDF metadata, page numbers, links, initial view mode, and remove open actions. -
25
LiteParse
A fast, helpful, and open-source document parser
LiteParse is an open-source lightweight parsing library designed to extract structured data from unstructured text using large language models in an efficient and cost-effective manner. It focuses on simplifying the process of turning raw text into structured outputs such as JSON by providing a streamlined interface for prompt-based parsing. The system is designed to minimize overhead, making it suitable for applications where performance and cost are critical considerations. ...
