Search Results for "data processing" - Page 3
Sort By:
Showing 418 open source projects for "data processing"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
1
Oban
Robust job processing in Elixir, backed by modern PostgreSQL
...It provides a simple and consistent API for scheduling and performing jobs, and it is built to be fault-tolerant and easy to monitor. Oban is fundamentally different from other background job processing tools because it retains job data for historic metrics and inspection. You can leave your application running indefinitely without worrying about jobs being lost or orphaned due to crashes. -
2
julep
A new DSL and server for AI agents and multi-step tasks
Julep is a platform for creating AI agents that remember past interactions and can perform complex tasks. It offers long-term memory and manages multi-step processes. Julep enables the creation of multi-step tasks incorporating decision-making, loops, parallel processing, and integration with numerous external tools and APIs. While many AI applications are limited to simple, linear chains of prompts and API calls with minimal branching, Julep is built to handle more complex scenarios. -
3
Smallpond
A lightweight data processing framework built on DuckDB and 3FS
smallpond is a lightweight distributed data processing framework built by DeepSeek, designed to scale DuckDB workloads over clusters using their 3FS (Fire-Flyer File System) backend. The idea is to preserve DuckDB’s fast analytics engine but lift it from single-node to multi-node settings, giving you the ability to operate on large datasets (e.g. petabyte scale) without moving to a heavyweight system like Spark. -
4
RuVector
Self-Learning, Vector Graph Neural Network, and Database built in Rust
RuVector is part of the broader rUv ecosystem of AI engineering tools and focuses on enabling advanced vector-based processing and intelligent system development within agentic and AI-driven pipelines. The project fits into a larger vision of modular, composable AI infrastructure designed to support autonomous agents, data retrieval, and intelligent automation workflows. It emphasizes extensibility and interoperability with modern AI stacks, allowing developers to integrate vector operations into search, reasoning, or generative systems. ... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
FFTW.jl
Julia bindings to the FFTW library for fast Fourier transforms
This package provides Julia bindings to the FFTW library for fast Fourier transforms (FFTs), as well as functionality useful for signal processing. These functions were formerly a part of Base Julia. Users with a build of Julia based on Intel's Math Kernel Library (MKL) can use MKL for FFTs by setting a preference in their top-level project by either using the FFTW.set_provider!() method, or by directly setting the preference using Preferences.jl. Note that this choice will be recorded for... -
6
Nuclio
High-Performance Serverless event and data processing platform
Nuclio is an open source and managed serverless platform used to minimize development and maintenance overhead and automate the deployment of data-science-based applications. Real-time performance running up to 400,000 function invocations per second. Portable across low laptops, edge, on-prem and multi-cloud deployments. The first serverless platform supporting GPUs for optimized utilization and sharing. Automated deployment to production in a few clicks from Jupyter notebook. Deploy one of... -
7
Colly
Elegant Scraper and Crawler Framework for Golang
Colly provides a clean interface to write any kind of crawler/scraper/spider. With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving. Clean API. Fast (>1k request/sec on a single core) Manages request delays and maximum concurrency per domain. Automatic cookie and session handling. Sync/async/parallel scraping. Distributed scraping. Caching, automatic encoding of non-unicode responses. ... -
8
Gooo
Toolkit for developing web applications in Vue, Templ, and Go
Gooo is an open-source project that focuses on providing tools and utilities for interacting with and analyzing online content, particularly in the context of automation and data retrieval workflows. The repository appears to function as a lightweight utility toolkit that can be adapted for specific use cases such as scraping, automation, or content processing. It is structured to allow developers to customize and extend its functionality depending on their needs, rather than acting as a fully packaged end-user application. ... -
9
ffmpeg.wasm
FFmpeg for browser, powered by WebAssembly
ffmpeg.wasm is a pure WebAssembly (and JavaScript/TypeScript) port of FFmpeg that enables in-browser media recording, conversion, and streaming—letting developers perform video/audio processing entirely client-side without server uploads. Transpiled via Emscripten from FFmpeg and its codecs into WebAssembly. Supports both single-threaded and multi-threaded cores using web workers. Written in TypeScript for improved developer experience. -
One Monitoring Tool for IT, OT and Cloud | Free TrialModern infrastructure spans data centers, cloud platforms and factory floors, and every blind spot between them is a risk. PRTG supports SNMP, WMI, SSH and other standard protocols to monitor IT, OT and hybrid environments through one customizable dashboard. Build the views your team needs, from network health to application performance, without switching tools. Try PRTG free for 30 days now.
-
10
ReductStore
The fastest time series object store for Edge AI
History storage and management of images, vibration data, text, labels, and more - all in one place with the highest performance. Merge blob and time series functionalities, reducing the need for multiple databases. Customize real-time data retention policies and replication strategies. Store billions of time-stamped blobs with AI labels and access them with low latency. Outperform other databases with a customized solution for time-series object data. Capture and access blob data as time... -
11
Google Highway
Performance-portable, length-agnostic SIMD with runtime dispatch
...This portability is achieved through dynamic or static dispatch mechanisms that select the best available instruction set at runtime or compile time. The library is designed for developers who need to maximize CPU performance in domains such as image processing, compression, cryptography, and scientific computing. -
12
baoyu-skills
Skills shared by Baoyu for improving daily work efficiency with Claude
baoyu-skills is a comprehensive plugin-based toolkit designed to enhance productivity within AI-assisted development environments, particularly Claude Code, by providing a wide range of modular “skills” for content generation, automation, and data processing. The project organizes its functionality into categories such as content creation, AI generation, and utility tools, enabling users to extend their workflows through reusable components. Each skill is implemented as a structured module with its own configuration, scripts, and execution logic, allowing for flexible customization and extension. ... -
13
Symfony DomCrawler
Eases DOM navigation for HTML and XML documents
Symfony DomCrawler is a PHP component that provides powerful tools for navigating and extracting data from HTML and XML documents. It allows developers to parse, filter, and manipulate web pages using CSS selectors and XPath expressions. DomCrawler is widely used for web scraping, testing, and processing structured content, and integrates well with other Symfony components like BrowserKit. -
14
Llama Cloud Services
Knowledge Agents and Management in the Cloud
Llama Cloud Services is a suite of tools designed to facilitate the integration of large language models (LLMs) into applications. It offers components for parsing, extracting, and reporting on complex documents, streamlining the process of preparing data for LLM consumption. -
15
Apache Bigtop
Bigtop is an Apache Foundation project for Infrastructure Engineers
...Developers and operators can use Bigtop to assemble customized Hadoop distributions tailored to their infrastructure and workloads. Its focus on reproducibility and packaging reduces friction in deploying large-scale data processing systems and ensures that different components of the Hadoop ecosystem work well together. -
16
Argo Workflows
Workflow engine for Kubernetes
...Model multi-step workflows as a sequence of tasks or capture the dependencies between tasks using a directed acyclic graph (DAG). Easily run compute intensive jobs for machine learning or data processing in a fraction of the time using Argo Workflows on Kubernetes. Run CI/CD pipelines natively on Kubernetes without configuring complex software development products. Argo Workflows is the most popular workflow execution engine for Kubernetes. It can run 1000s of workflows a day, each with 1000s of concurrent tasks. Our users say it is lighter-weight, faster, more powerful, and easier to use. ... -
17
Polars
Dataframes powered by a multithreaded, vectorized query engine
Polars is a high-performance, multi-language DataFrame library built in Rust using Apache Arrow. It delivers blazing-fast, vectorized, and parallel data manipulation with both eager and lazy execution, making it an excellent tool for data processing in Python, Rust, Node.js, R, and SQL contexts. -
18
CUDA Python
Performance meets Productivity
...The toolkit also includes utilities for profiling, memory management, distributed computing, and numerical operations, making it suitable for scientific computing, AI, and data processing workloads. -
19
Gson
A Java serialization/deserialization library to convert Java Objects
Gson is a Java library developed by Google that allows conversion between Java objects and JSON. It enables serialization and deserialization of Java classes to and from JSON format, handling complex and generic types, nulls, custom naming policies, and more. Gson is lightweight, easy to use, and does not require annotation-based configuration, making it a popular choice for JSON processing in Java applications. -
20
ElasticJob
Distributed scheduled job framework
ElasticJob is a distributed scheduling solution consisting of two separate projects, ElasticJob-Lite and ElasticJob-Cloud. ElasticJob-Lite is a lightweight, decentralized solution that provides distributed task sharding services. ElasticJob-Cloud uses Mesos to manage and isolate resources. It uses a unified job API for each project. Developers only need code one time and can deploy at will. Support job sharding and high availability in distributed system. Scale out for throughput and... -
21
GeoStats.jl
An extensible framework for geospatial data science
GeoStats.jl is a Julia framework for geospatial data science and geostatistical modeling. It’s fully implemented in Julia and designed to provide an extensible, high-performance stack that handles spatial domains, interpolation, simulation, learning, and visualization. The package is modular: it breaks out geometry, spatial domains, transforms, variograms, covariance models, and modeling into subpackages (e.g., GeoStatsBase, GeoStatsModels, GeoStatsTransforms). Users can represent... -
22
Google Cloud Platform Python Samples
Code samples used on cloud.google
...It serves as a practical companion to official documentation, providing runnable snippets that illustrate how to authenticate, configure environments, and interact with APIs across products such as storage, AI services, and data processing tools. The repository is organized into product-specific directories, allowing developers to quickly locate examples relevant to their use case and adapt them into production workflows. It emphasizes hands-on learning by guiding users through setup steps such as creating virtual environments, installing dependencies, and running scripts locally. ... -
23
OfficeCLI
OfficeCLI is the first and best command-line tool
OfficeCLI is a command-line productivity tool designed to bring AI-powered automation into everyday office workflows, enabling users to perform tasks such as document generation, data processing, and communication management directly from the terminal. It focuses on simplifying repetitive business operations by translating natural language commands into structured actions. The system likely integrates with common office tools and formats, allowing seamless interaction with documents, spreadsheets, and communication platforms. ... -
24
ReactPHP Promise
Promises/A implementation for PHP.
ReactPHP Promise is a PHP library that implements Promises/A+ for asynchronous programming. It allows developers to handle asynchronous operations in a structured and predictable way. ReactPHP Promise makes it easier to manage non-blocking tasks, such as HTTP requests or file I/O, by providing a chainable interface for handling success and error cases. -
25
OnlyOffice Web
Perform common file preview and editing via the web
OnlyOffice Web is a browser-based document editing platform built on top of OnlyOffice that allows users to view and edit files entirely on the client side without requiring a backend server. It is designed with a privacy-first approach, ensuring that all document processing occurs locally in the browser, which prevents sensitive data from being uploaded or stored externally. The application supports a wide range of file formats, including DOCX, XLSX, PPTX, and CSV, making it versatile for handling common office documents. It offers a smooth and responsive editing experience, leveraging modern web technologies to replicate the functionality of traditional desktop office suites. ...
