Search Results for "data extract" - Page 2
Sort By:
Showing 623 open source projects for "data extract"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
JSLT
JSON query and transformation language
JSLT is a complete query and transformation language for JSON. The language design is inspired by jq, XPath, and XQuery. -
2
ML for Trading
Code for machine learning for algorithmic trading, 2nd edition
On over 800 pages, this revised and expanded 2nd edition demonstrates how ML can add value to algorithmic trading through a broad range of applications. Organized in four parts and 24 chapters, it covers the end-to-end workflow from data sourcing and model development to strategy backtesting and evaluation. Covers key aspects of data sourcing, financial feature engineering, and portfolio management. The design and evaluation of long-short strategies based on a broad range of ML algorithms, how to extract tradeable signals from financial text data like SEC filings, earnings call transcripts or financial news. ... -
3
AI-Crawler
Crawl a website starting from a URL, find relevant pages
...Users can define their data requirements in plain English, and the system will interpret those instructions to crawl a domain and extract structured data. The tool supports output formats such as JSON and Markdown, and it can generate or accept schemas to ensure that extracted data is structured according to application needs. It is designed as a low-code solution, reducing the complexity of building and maintaining custom scraping pipelines. -
4
DouK-Downloader
TikTok releases/likes/compilations/live streams/videos/atlases/music
DouK-Downloader is a fully open-source data acquisition and media downloading tool designed to extract, collect, and download content from TikTok and its Chinese counterpart Douyin at scale. Built using Python and modern asynchronous networking libraries such as HTTPX, it enables batch downloading of videos, images, live streams, and metadata from accounts, playlists, and individual links. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
Spider
High-performance Rust web crawler and scraper for large-scale data
...These capabilities make the project suitable for building search indexers, data extraction pipelines, & SEO analysis tools. -
6
PDFCraft
PDFCraft is a free, privacy-focused PDF toolkit
...At its core, the project provides a clean, modern UI where you can rearrange pages, annotate text, insert images, fill forms, and export to multiple formats, all without needing a heavyweight commercial PDF suite. But beyond manual editing, it also offers a programmable layer so developers can write scripts to batch process documents, generate templated reports, or extract structured data from PDFs for integration in workflows. The design emphasizes quality and compatibility: output PDFs render accurately across readers, preserve metadata, and support interactive elements like hyperlinks and form fields. -
7
watercrawl
AI-ready web crawler that extracts and structures website content
WaterCrawl is an open source web crawling and data extraction platform designed to transform website content into structured data suitable for machine learning and AI workflows. It enables developers and researchers to crawl web pages, extract meaningful information, and convert it into formats that are easier to process and analyze. It provides a modern crawling system that can automatically navigate links, control crawl depth, and collect content from targeted sections of a website. ... -
8
DiscordChatExporter
Saves Discord chat logs to a file
...Overall, DiscordChatExporter provides a reliable way to extract and preserve Discord communications outside the platform. -
9
go-i18n
Translate your Go program into multiple languages
go-i18n is a Go package and a command that helps you translate Go programs into multiple languages. Supports pluralized strings for all 200+ languages in the Unicode Common Locale Data Repository (CLDR). Code and tests are automatically generated from CLDR data. Supports strings with named variables using text/template syntax. Supports message files of any format (e.g. JSON, TOML, YAML). Use goi18n extract to extract all i18n.Message struct literals in Go source files to a message file for translation. Create an empty message file for the language that you want to add (e.g. translate.es.toml). ... -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
newpipeextractor
Library for extracting streaming site data without official APIs
NewPipeExtractor is an open source Java library designed to extract data from streaming platforms by analyzing their web interfaces instead of relying on official APIs. It serves as the core extraction component used by the NewPipe Android application, but it is built as a standalone library that can also be integrated into other software projects. NewPipeExtractor provides a unified framework for retrieving information such as video streams, playlists, channels, and search results from supported streaming services. ... -
11
cognee
Deterministic LLMs Outputs for AI Applications and AI Agents
...Any kind of data works; unstructured text or raw media files, PDFs, tables, presentations, JSON files, and so many more. Add small or large files, or many files at once. We map out a knowledge graph from all the facts and relationships we extract from your data. Then, we establish graph topology and connect related knowledge clusters, enabling the LLM to "understand" the data. -
12
GoldenCheetah
Performance Software for Cyclists, Runners, Triathletes and Coaches
Analyze using summary metrics like BikeStress, TRIMP, or RPE. Extract insight via models like Critical Power and W'bal. Track and predict performance using models like Banister and PMC. Optimize aerodynamics using Virtual Elevation. Train indoors with ANT and BTLE trainers. Upload and Download with many cloud services including Strava, Withings, and Today's Plan. Import and export data to and from a wide range of bike computers and file formats. -
13
DataProfiler
Extract schema, statistics and entities from datasets
DataProfiler is an AI-powered tool for automatic data analysis and profiling, designed to detect patterns, anomalies, and schema inconsistencies in structured and unstructured datasets. The DataProfiler is a Python library designed to make data analysis, monitoring, and sensitive data detection easy. Loading Data with a single command, the library automatically formats & loads files into a DataFrame. Profiling the Data, the library identifies the schema, statistics, entities (PII / NPI), and... -
14
JSONPath Plus
A fork of JSONPath
Analyse, transform, and selectively extract data from JSON documents (and JavaScript objects). JSON path-plus expands on the original specification to add some additional operators and makes explicit some behaviors the original did not spell out. Try the browser demo or Runkit (Node). -
15
esquisse
RStudio add-in to make plots interactively with ggplot2
The purpose of this add-in is to let you explore your data quickly to extract the information they hold. You can create visualization with {ggplot2}, filter data with {dplyr} and retrieve generated code. This addin allows you to interactively explore your data by visualizing it with the ggplot2 package. It allows you to draw bar plots, curves, scatter plots, histograms, boxplot and sf objects, then export the graph or retrieve the code to reproduce the graph. ... -
16
Alternative JSON parser for Go
One of the fastest alternative JSON parser for Go
It does not require you to know the structure of the payload (eg. create structs), and allows accessing fields by providing the path to them. It is up to 10 times faster than standard encoding/json package (depending on payload size and usage), allocates no memory. Originally made for a project that relies on a lot of 3rd party APIs that can be unpredictable and complex. Encoding/json requires you to know exactly your data structures, or if you prefer to use map[string]interface{} instead,... -
17
jsoup
Java library for working with real-world HTML
jsoup is a Java library for working with real-world HTML. It provides a very convenient API for fetching URLs and extracting and manipulating data, using the best of HTML5 DOM methods and CSS selectors. jsoup implements the WHATWG HTML5 specification, and parses HTML to the same DOM as modern browsers do. jsoup is designed to deal with all varieties of HTML found in the wild; from pristine and validating, to invalid tag-soup; jsoup will create a sensible parse tree. The parser will make... -
18
Weibo Crawler
Python crawler for collecting and downloading Sina Weibo user data
weibo-crawler is a Python-based data collection tool designed to retrieve information from Sina Weibo user accounts. It automates the process of gathering posts, user profile details, and engagement metrics from one or more target accounts. weibo-crawler can extract comprehensive information about users, including profile attributes such as nickname, follower count, following count, and account metadata. -
19
docext
An on-premises, OCR-free unstructured data extraction
docext is a document intelligence toolkit that uses vision-language models to extract structured information from documents such as PDFs, forms, and scanned images. The system is designed to operate entirely on-premises, allowing organizations to process sensitive documents without relying on external cloud services. Unlike traditional document processing pipelines that rely heavily on optical character recognition, docext leverages multimodal AI models capable of understanding both visual... -
20
Chandra
OCR model for complex documents with layout-aware structured outputs
Chandra is an advanced OCR model designed to extract and structure information from complex documents such as tables, forms, handwritten notes, and mathematical content. It focuses on preserving full document layout, meaning that extracted text is accompanied by positional metadata like bounding boxes for each element. Chandra supports multiple output formats including Markdown, HTML, and JSON, making it suitable for downstream processing and integration into data pipelines. ... -
21
Jbuilder
Generate JSON objects with a Builder-style DSL
Jbuilder gives you a simple DSL for declaring JSON structures that beats manipulating giant hash structures. This is particularly helpful when the generation process is fraught with conditionals and loops. You can either use Jbuilder stand-alone or directly as an ActionView template language. When required in Rails, you can create views à la show.json.jbuilder (the json is already yielded). Fragment caching is supported, it uses Rails.cache and works like caching in HTML templates. If your... -
22
BettaFish
Public opinion analysis system
BettaFish is an open-source, multi-agent public opinion analysis system built to automate the collection, deep analysis, and reporting of social media data at scale through conversational queries. It uses a modular architecture of specialized agents that collaborate to crawl mainstream platforms, extract multimodal content like text and short video, and synthesize insights through both statistical and large language model techniques. With a design that lets users pose questions in natural language and receive structured reports, charts, and visualizations, the system aims to break information cocoons and provide comprehensive views of trends and public sentiment. ... -
23
monolith
CLI tool for saving complete web pages as a single HTML file
A data hoarder’s dream come true, bundle any web page into a single HTML file. You can finally replace that gazillion of open tabs with a gazillion of .html files stored somewhere on your precious little drive. Unlike the conventional “Save page as”, monolith not only saves the target document, it embeds CSS, image, and JavaScript assets all at once, producing a single HTML5 document that is a joy to store and share. If compared to saving websites with wget -mpk, this tool embeds all assets... -
24
dovi_tool
dovi_tool is a CLI tool combining multiple utilities
dovi_tool is a command-line utility written in Rust that provides a comprehensive set of tools for working with Dolby Vision metadata in video streams. It is designed to analyze, edit, and generate dynamic metadata used in high dynamic range video formats. The tool allows users to extract, inject, and modify RPU data, which controls how Dolby Vision content is displayed on compatible devices. It also supports demuxing and muxing HEVC streams, enabling manipulation of enhancement layers and metadata within video files. dovi_tool is widely used in advanced video encoding workflows where precise control over HDR metadata is required. ... -
25
Figma Sprite Generator
A Figma plugin to generate sprite sheets and JSON files from selected
Figma Sprite Generator is a Figma plugin designed to create sprite sheets and corresponding JSON metadata from selected icons. This tool is ideal for designers and developers working on web projects or UI libraries, as it automates sprite generation for more efficient workflows. Users can quickly convert their icons into PNG sprites and extract JSON files with icon dimensions and positions, streamlining the process of incorporating sprites into websites or applications.
