Search Results for "data processing" - Page 3
Sort By:
Showing 359 open source projects for "data processing"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
1
Video-subtitle-remover (VSR)
AI tool that removes hardcoded subtitles and text from videos locally
...It allows users to define a specific subtitle region so that only text in that area is removed rather than modifying the entire frame. It can also automatically remove text throughout the whole video when a position is not specified. In addition to video processing, the project supports removing text-like watermarks from images through similar techniques. The processing runs locally without requiring any external API services, enabling offline use and greater control over the data being processed. -
2
paperless-gpt
Use LLMs and LLM Vision (OCR) to handle paperless-ngx
paperless-gpt is an AI-powered extension for document management systems that enhances the capabilities of paperless-ngx by integrating large language models and vision-based OCR to automate document processing and organization. It is designed to transform scanned or uploaded documents into structured, searchable, and intelligently categorized data without requiring manual tagging or sorting. The system uses OCR combined with LLM reasoning to extract text, classify documents, and generate metadata such as tags, titles, and categories automatically. ... -
3
compromise
Modest natural-language processing
Language is complicated and there's a gazillion words. Compromise is a javascript library that interprets and pre-parses text and makes some reasonable decisions so things are way easier. Compromise tries its best to parse text. it is small, quick, and often good-enough. It is not as smart as you'd think. Conjugate and negate verbs in any tense. Play between plural, singular and possessive forms. Interpret plain-text numbers. Handle implicit terms. Use it on the client-side or as an... -
4
Pluely
The Open Source Alternative to Cluely
...The system focuses on orchestrating tasks performed by large language models and other AI components, allowing developers to define structured workflows where models interact with tools, APIs, and external systems. By providing a modular architecture for building AI pipelines, the platform enables developers to connect multiple processing steps such as data retrieval, prompt execution, analysis, and response generation. The project emphasizes flexibility, allowing developers to extend the platform with custom integrations and automation logic. This makes the framework suitable for building intelligent assistants, automated business workflows, and data-processing pipelines that rely on generative AI capabilities. -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
5
BambooAI
A Python library powered by Language Models (LLMs)
BambooAI is a Python library powered by large language models (LLMs) for conversational data discovery and analysis, allowing users to interact with data through natural language. -
6
Chandra
OCR model for complex documents with layout-aware structured outputs
...It focuses on preserving full document layout, meaning that extracted text is accompanied by positional metadata like bounding boxes for each element. Chandra supports multiple output formats including Markdown, HTML, and JSON, making it suitable for downstream processing and integration into data pipelines. It is capable of handling over 40 languages and is optimized to read difficult inputs such as messy handwriting and multi-column layouts. Chandra can be run locally using transformer-based inference or deployed with a high-performance server setup for large-scale processing. It also includes command-line tools and optional web-based interfaces to simplify interaction and batch processing workflows. -
7
Quantitative Trading System
A comprehensive quantitative trading system with AI-powered analysis
Quantitative Trading System is a comprehensive quantitative trading platform that integrates artificial intelligence, financial data analysis, and automated strategy execution within a unified software system. The project is designed to provide an end-to-end infrastructure for building and operating algorithmic trading strategies in financial markets. It includes tools for collecting and processing market data from multiple sources, performing statistical and machine learning analysis, and generating trading signals based on quantitative models. ... -
8
Spark NLP
State of the Art Natural Language Processing
...The estimators have a method that secures and trains a piece of data to such an application. The transformer is generally the result of a fitting process and applies changes to the target dataset. These components have been embedded to be applicable to Spark NLP. Pipelines are a mechanism for combining multiple estimators and transformers in a single workflow. They allow multiple chained transformations along a machine-learning task. -
9
FIT Framework
An enterprise-level AI development framework
FIT Framework is an open-source infrastructure designed to support the development, training, and evaluation of machine learning and AI models through a modular and scalable architecture. It aims to streamline the lifecycle of AI systems by providing standardized components for data processing, model training, evaluation, and deployment. The framework is particularly useful for research and production environments where reproducibility and consistency are critical, as it enforces structured workflows and configurable pipelines. It supports experimentation with different models and datasets, allowing developers to iterate quickly while maintaining clear organization of results and configurations. ... -
Try Google Cloud Risk-Free With $300 in CreditUse your credit across every product. Compute, storage, AI, analytics. When it runs out, 20+ products stay free. You only pay when you choose to.
-
10
TorchAudio
Data manipulation and transformation for audio signal processing
The aim of torchaudio is to apply PyTorch to the audio domain. By supporting PyTorch, torchaudio follows the same philosophy of providing strong GPU acceleration, having a focus on trainable features through the autograd system, and having consistent style (tensor names and dimension names). Therefore, it is primarily a machine learning library and not a general signal processing library. The benefits of PyTorch can be seen in torchaudio through having all the computations be through PyTorch... -
11
ESPnet
End-to-end speech processing toolkit
ESPnet is a comprehensive end-to-end speech processing toolkit covering a wide spectrum of tasks, including automatic speech recognition (ASR), text-to-speech (TTS), speech translation (ST), speech enhancement, speaker diarization, and spoken language understanding. It uses PyTorch as its deep learning engine and adopts a Kaldi-style data processing pipeline for features, data formats, and experimental recipes. -
12
Taipy
Turns Data and AI algorithms into production-ready web applications
...Taipy enhances performance with caching control of graphical events, optimizing rendering by selectively updating graphical components only upon interaction. Effortlessly manage massive datasets with Taipy's built-in decimator for charts, intelligently reducing the number of data points to save time and memory without losing the essence of your data's shape. Struggle with sluggish performance and excessive memory usage, as every data point demands processing. Large datasets become cumbersome, complicating the user experience and data analysis. Scenarios are made easy with Taipy Studio. A powerful VS Code extension that unlocks a convenient graphical editor. ... -
13
Open Semantic Search
Open source semantic search and text analytics for large document sets
...It integrates text mining and analytics capabilities that allow users to examine relationships, topics, and structured data within document collections. -
14
Browser Use
Make websites accessible for AI agents
Browser-Use is a framework that makes websites accessible for AI agents, enabling automated interactions and data extraction from web pages. -
15
Nexent
Zero-code platform for building AI agents from natural language input
...It focuses on a zero-code approach, allowing users to define workflows and agent behavior purely through language prompts, significantly lowering the barrier to entry for AI development. Built on the MCP ecosystem, Nexent integrates a wide range of tools, models, and data sources into a unified environment for agent creation and execution. Nexent supports multi-agent collaboration, enabling multiple intelligent agents to interact and coordinate tasks within complex workflows. It also includes capabilities for data processing, knowledge tracing, and multimodal interaction, allowing agents to work with different input and output formats. ... -
16
Matrix
Multi-Agent daTa geneRation Infra and eXperimentation framework
Matrix is a distributed, large-scale engine for multi-agent synthetic data generation and experiments: it provides the infrastructure to run thousands of “agentic” workflows concurrently (e.g. multiple LLMs interacting, reasoning, generating content, data-processing pipelines) by leveraging distributed computing (like Ray + cluster management). The idea is to treat data generation as a “data-to-data” transformation: each input item defines a task, and the runtime orchestrates asynchronous, peer-to-peer agent workflows, avoiding global synchronization bottlenecks. ... -
17
MegaParse
File Parser optimised for LLM Ingestion with no loss
MegaParse is a file parser optimized for Large Language Model (LLM) ingestion, ensuring no loss of information. It efficiently parses various document formats, such as PDFs, DOCX, and PPTX, converting them into formats ideal for processing by LLMs. This tool is essential for applications that require accurate and comprehensive data extraction from diverse document types. -
18
AutoGluon
AutoGluon: AutoML for Image, Text, and Tabular Data
AutoGluon enables easy-to-use and easy-to-extend AutoML with a focus on automated stack ensembling, deep learning, and real-world applications spanning image, text, and tabular data. Intended for both ML beginners and experts, AutoGluon enables you to quickly prototype deep learning and classical ML solutions for your raw data with a few lines of code. Automatically utilize state-of-the-art techniques (where appropriate) without expert knowledge. Leverage automatic hyperparameter tuning, model selection/ensembling, architecture search, and data processing. ... -
19
NeMo Retriever Library
Document content and metadata extraction microservice
...It processes various document types by splitting them into components such as text, tables, charts, and images, and then applies OCR and contextual analysis to convert them into structured data formats. The system is built on NVIDIA NIM microservices, enabling high-performance parallel processing and efficient handling of large datasets. It supports multiple extraction strategies for different document formats, balancing accuracy and throughput depending on the use case. Additionally, it can generate embeddings for extracted content and integrate with vector databases like Milvus, making it well-suited for retrieval-augmented generation pipelines. -
20
Haystack
Haystack is an open source NLP framework to interact with your data
Apply the latest NLP technology to your own data with the use of Haystack's pipeline architecture. Implement production-ready semantic search, question answering, summarization and document ranking for a wide range of NLP applications. Evaluate components and fine-tune models. Ask questions in natural language and find granular answers in your documents using the latest QA models with the help of Haystack pipelines. Perform semantic search and retrieve ranked documents according to meaning,... -
21
Lingua-Py
The most accurate natural language detection library for Python
Its task is simple: It tells you which language some text is written in. This is very useful as a preprocessing step for linguistic data in natural language processing applications such as text classification and spell checking. Other use cases, for instance, might include routing e-mails to the right geographically located customer service department, based on the e-mails' languages. Language detection is often done as part of large machine learning frameworks or natural language processing applications. ... -
22
BettaFish
Public opinion analysis system
...Unlike simpler analytics tools, BettaFish employs agent collaboration and a “forum” style internal mechanism to combine diverse model outputs, making the analysis richer and more robust. It also integrates multimodal processing, enabling it to parse images and video alongside text. -
23
nesa
Run AI models end-to-end encrypted
nesa is an open-source initiative focused on building decentralized AI infrastructure that enables secure, verifiable, and privacy-preserving machine learning and inference across distributed environments. The project aims to address key challenges in modern AI systems, such as data privacy, trust, and centralization, by leveraging cryptographic techniques and decentralized architectures. NESA allows developers to run AI computations in a way that ensures data integrity and confidentiality,... -
24
airda
airda(Air Data Agent
airda(Air Data Agent) is a multi-smart body for data analysis, capable of understanding data development and data analysis needs, understanding data, generating data-oriented queries, data visualization, machine learning and other tasks of SQL and Python codes. -
25
docext
An on-premises, OCR-free unstructured data extraction
docext is a document intelligence toolkit that uses vision-language models to extract structured information from documents such as PDFs, forms, and scanned images. The system is designed to operate entirely on-premises, allowing organizations to process sensitive documents without relying on external cloud services. Unlike traditional document processing pipelines that rely heavily on optical character recognition, docext leverages multimodal AI models capable of understanding both visual...
