Search Results for "data processing" - Page 4
Sort By:
Showing 429 open source projects for "data processing"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
Build Securely on Azure with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
1
Windrecorder
Windrecorder is a memory search app by records everything
Windrecorder is an open-source personal memory search engine that continuously records on-screen activity in a highly optimized and storage-efficient format. It captures screen content locally and builds a searchable database using OCR and image understanding, allowing users to rewind and rediscover anything they have previously seen. The system indexes only meaningful visual changes, extracting text, browser data, and contextual information to improve search accuracy and reduce storage... -
2
NVIDIA NeMo
Toolkit for conversational AI
NVIDIA NeMo, part of the NVIDIA AI platform, is a toolkit for building new state-of-the-art conversational AI models. NeMo has separate collections for Automatic Speech Recognition (ASR), Natural Language Processing (NLP), and Text-to-Speech (TTS) models. Each collection consists of prebuilt modules that include everything needed to train on your data. Every module can easily be customized, extended, and composed to create new conversational AI model architectures. Conversational AI architectures are typically large and require a lot of data and compute for training. ... -
3
CogDB
Micro Graph Database for Python Applications
Cog is a lightweight, embedded graph database for Go that provides a simple interface for storing and querying graph-based data structures, making it useful for knowledge representation and graph analytics. -
4
Llama Cloud Services
Knowledge Agents and Management in the Cloud
Llama Cloud Services is a suite of tools designed to facilitate the integration of large language models (LLMs) into applications. It offers components for parsing, extracting, and reporting on complex documents, streamlining the process of preparing data for LLM consumption. -
PRTG Catches Network Issues Before They Cause DowntimeReactive troubleshooting usually means hearing about a problem from frustrated users, not your monitoring tool. PRTG sets threshold-based alerts across devices, servers and applications, notifying your team by email, SMS or push the moment a metric crosses a set limit. That means catching a failing disk or overloaded server before it becomes an outage and getting time back from firefighting. Start a free trial and set your first alerts today.
-
5
OpenRecall
OpenRecall is a fully open-source, privacy-first alternative
OpenRecall is an open-source, privacy-first system designed to capture, index, and make searchable a user’s entire digital activity history, effectively acting as a personal memory layer for computing environments. It works by taking periodic screenshots of a user’s screen and applying local AI processing, including OCR and semantic analysis, to extract and structure information from both text and images. This data is then indexed into a searchable database, allowing users to retrieve past information quickly using natural language queries. Unlike proprietary alternatives, OpenRecall operates entirely locally, ensuring that all captured data remains on the user’s device and is never transmitted to external servers. ... -
6
SuperDuperDB
Integrate, train and manage any AI models and APIs with your database
...Integrate and combine models from Sklearn, PyTorch, HuggingFace with AI APIs such as OpenAI to build even the most complex AI applications and workflows. Train models on your data in your datastore simply by querying without additional ingestion and pre-processing. -
7
Trafilatura
Python & command-line tool to gather text on the Web
Trafilatura is a Python package and command-line tool designed to gather text on the Web. It includes discovery, extraction and text-processing components. Its main applications are web crawling, downloads, scraping, and extraction of main texts, metadata and comments. It aims at staying handy and modular: no database is required, the output can be converted to various commonly used formats. Going from raw HTML to essential parts can alleviate many problems related to text quality, first by avoiding the noise caused by recurring elements (headers, footers, links/blogroll etc.) and second by including information such as author and date in order to make sense of the data. ... -
8
Chandra
OCR model for complex documents with layout-aware structured outputs
...It focuses on preserving full document layout, meaning that extracted text is accompanied by positional metadata like bounding boxes for each element. Chandra supports multiple output formats including Markdown, HTML, and JSON, making it suitable for downstream processing and integration into data pipelines. It is capable of handling over 40 languages and is optimized to read difficult inputs such as messy handwriting and multi-column layouts. Chandra can be run locally using transformer-based inference or deployed with a high-performance server setup for large-scale processing. It also includes command-line tools and optional web-based interfaces to simplify interaction and batch processing workflows. -
9
Python API for JMComic
Python crawler and API for downloading JMComic albums and images
...It provides a structured API that allows developers to retrieve albums, chapters, and images using simple Python code while handling the necessary network requests and data processing behind the scenes. It supports both web-based and mobile API interfaces, enabling flexible interaction with the platform depending on the available endpoints. Its architecture includes components for configuration management, download orchestration, and client communication, allowing users to automate the retrieval of manga chapters or entire albums. ... -
Demo Series - Small Business Backup By VeeamWatch this on-demand demo series and learn how to protect your Microsoft 365 data with clear, simple, actionable steps that are easy to implement for businesses of all sizes.
-
10
Docling
Get your documents ready for gen AI
Docling is an open-source document processing toolkit built to prepare diverse content types for modern generative AI and data workflows. The project focuses on converting and parsing many document formats into a unified structured representation that downstream systems can easily consume. It supports advanced PDF understanding, including layout detection, table extraction, and reading order analysis, enabling high-fidelity document intelligence pipelines. -
11
The FreeMoCap Project
Free Motion Capture for Everyone
...FreeMoCap includes tools for calibration, recording, processing, and visualization, allowing users to move from raw footage to structured motion data within a single ecosystem. Because it is open and extensible, researchers and developers can adapt the pipeline for specialized motion analysis or integrate it into animation workflows. -
12
Milvus Bootcamp
Dealing with all unstructured data, such as reverse image search
Milvus Bootcamp is a collection of tutorials, examples, and best practices for using Milvus, an open-source vector database designed for AI-powered similarity search and retrieval applications. -
13
POT
Python Optimal Transport
This open source Python library provides several solvers for optimization problems related to Optimal Transport for signal, image processing and machine learning. -
14
MCP Neo4j
Model Context Protocol with Neo4j
An implementation of the Model Context Protocol with Neo4j, enabling natural language interactions with Neo4j databases and facilitating operations such as schema retrieval and Cypher query execution. -
15
Advanced NLP with spaCy
Advanced NLP with spaCy: A free online course
...It also demonstrates how spaCy pipelines work and how developers can extend them with custom components and training data. The course is structured as a hands-on learning environment where students can run code examples, experiment with NLP techniques, and build practical language processing applications. Because spaCy is widely used in production environments, the course emphasizes industrial-strength NLP workflows and best practices. -
16
NeMo Curator
Scalable data pre processing and curation toolkit for LLMs
NeMo Curator is a Python library specifically designed for fast and scalable dataset preparation and curation for large language model (LLM) use-cases such as foundation model pretraining, domain-adaptive pretraining (DAPT), supervised fine-tuning (SFT) and paramter-efficient fine-tuning (PEFT). It greatly accelerates data curation by leveraging GPUs with Dask and RAPIDS, resulting in significant time savings. The library provides a customizable and modular interface, simplifying pipeline... -
17
CUDA Python
Performance meets Productivity
...The toolkit also includes utilities for profiling, memory management, distributed computing, and numerical operations, making it suitable for scientific computing, AI, and data processing workloads. -
18
Parser-Chitai-Gorod
Intelligent parser for catalog «Read-city»
Processing can use multiple threads to improve throughput across larger catalogs. Completed URLs are tracked so interrupted jobs can resume without unnecessarily processing the same books again. Results are exported to Excel workbooks, with separate sheets organized by book genre for easier analysis. -
19
Viral-Clips-Crew
Your CrewAI Powered Video Editing Assistant
Viral-Clips-Crew is an AI-driven video processing pipeline designed to generate short-form, engaging clips from long-form video content automatically. It analyzes transcripts and video data to identify the most engaging or “viral” moments, reducing the need for manual editing. The system integrates tools like FFmpeg and AI models to handle segmentation, cropping, and formatting for vertical video platforms. -
20
NeMo Retriever Library
Document content and metadata extraction microservice
...It processes various document types by splitting them into components such as text, tables, charts, and images, and then applies OCR and contextual analysis to convert them into structured data formats. The system is built on NVIDIA NIM microservices, enabling high-performance parallel processing and efficient handling of large datasets. It supports multiple extraction strategies for different document formats, balancing accuracy and throughput depending on the use case. Additionally, it can generate embeddings for extracted content and integrate with vector databases like Milvus, making it well-suited for retrieval-augmented generation pipelines. -
21
AI Powered Knowledge Graph Generator
AI Powered Knowledge Graph Generator
AI-Powered Knowledge Graph is an open-source project focused on building knowledge graph systems that integrate artificial intelligence and machine learning to represent complex relationships between data entities. Knowledge graphs organize information as networks of nodes and relationships, allowing applications to analyze connections between concepts, datasets, or real-world entities. By incorporating AI techniques such as natural language processing and semantic reasoning, the project enables systems to automatically extract relationships and insights from large volumes of data. -
22
python-socketio
Python Socket.IO server and client
python-socketio is a robust Python library that implements the Socket.IO protocol, enabling real-time, bidirectional communication between web clients and servers. It works with multiple asynchronous frameworks such as asyncio, eventlet, and gevent, so developers can choose the concurrency model that best fits their application needs while still using a consistent API. The library provides both server and client implementations, allowing Python applications to serve and communicate with... -
23
Google Cloud Platform Python Samples
Code samples used on cloud.google
...It serves as a practical companion to official documentation, providing runnable snippets that illustrate how to authenticate, configure environments, and interact with APIs across products such as storage, AI services, and data processing tools. The repository is organized into product-specific directories, allowing developers to quickly locate examples relevant to their use case and adapt them into production workflows. It emphasizes hands-on learning by guiding users through setup steps such as creating virtual environments, installing dependencies, and running scripts locally. ... -
24
douyin
Open source Douyin crawler for collecting and downloading public data
DouyinCrawler is an open source data collection tool designed to gather publicly available information from the Douyin platform. It demonstrates how to build a Python-based web crawler combined with a graphical interface and command line functionality. It allows users to collect data from various types of Douyin content, including user profiles, videos, hashtags, and music pages. DouyinCrawler supports both automated scraping and batch operations to process multiple targets efficiently. It... -
25
DATAGEN
AI-driven multi-agent research assistant automating hypothesis
...The project integrates several modern AI frameworks including LangChain, LangGraph, and large language models to manage reasoning and data processing tasks. Through this architecture, the system can combine structured data analysis with natural language reasoning to generate insights and research outputs. The platform is designed for researchers, analysts, and developers who want to accelerate data exploration and automate parts of the research lifecycle.
