Search Results for "data processing" - Page 2
Sort By:
Showing 455 open source projects for "data processing"
View related business solutions-
Earn up to 16% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
1
Elixir
Dynamic, functional language designed for building scalable apps
...Elixir leverages the Erlang VM, known for running low-latency, distributed, and fault-tolerant systems. Elixir is successfully used in web development, embedded software, data ingestion, and multimedia processing, across a wide range of industries. All Elixir code runs inside lightweight threads of execution (called processes) that are isolated and exchange information via messages. Due to their lightweight nature, it is not uncommon to have hundreds of thousands of processes running concurrently in the same machine. ... -
2
Jimp
An image processing library written entirely in JavaScript for Node
An image processing library for Node written entirely in JavaScript, with zero native dependencies. If you're using this library with TypeScript the method of importing slightly differs from JavaScript. Instead of using require, you must import it with ES6 default import scheme. If you're using a web bundles (webpack, rollup, parcel) you can benefit from using the module build of jimp. Using the module build will allow your bundler to understand your code better and exclude things you aren't... -
3
sharp
High performance Node.js image processing module
The typical use case for this high speed Node.js module is to convert large images in common formats to smaller, web-friendly JPEG, PNG, AVIF and WebP images of varying dimensions. Resizing an image is typically 4x-5x faster than using the quickest ImageMagick and GraphicsMagick settings due to its use of libvips. Colour spaces, embedded ICC profiles and alpha transparency channels are all handled correctly. Lanczos resampling ensures quality is not sacrificed for speed. As well as image... -
4
.NET for Apache Spark
A free, open-source, and cross-platform big data analytics framework
.NET for Apache Spark provides high-performance APIs for using Apache Spark from C# and F#. With these .NET APIs, you can access the most popular Dataframe and SparkSQL aspects of Apache Spark, for working with structured data, and Spark Structured Streaming, for working with streaming data. .NET for Apache Spark is compliant with .NET Standard - a formal specification of .NET APIs that are common across .NET implementations. This means you can use .NET for Apache Spark anywhere you write... -
Gemini 3 and 200+ AI Models on One PlatformBuild generative AI apps with Vertex AI. Switch between models without switching platforms.
-
5
WebP Codec
Library to encode and decode images in WebP format
libwebp is the reference codec library for Google’s WebP image format, providing both encoding and decoding along with command-line tools. It supplies cwebp to compress images into WebP and dwebp to decompress them back, making it easy to test quality/size trade-offs across presets and tuning parameters. The GitHub repository is a mirror; the canonical source of truth lives on Chromium’s git, and developer docs are hosted on WebP’s portal. The project underpins WebP support across browsers,... -
6
Spring Batch
Spring Batch is a framework for writing batch applications using Java
A lightweight, comprehensive batch framework designed to enable the development of robust batch applications vital for the daily operations of enterprise systems. Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging/tracing, transaction management, job processing statistics, job restart, skip, and resource management. It also provides more advanced technical services and features that will enable extremely high-volume and high... -
7
Apache Sedona
Cluster computing framework for processing large-scale geospatial data
Apache Sedona™ is a cluster computing system for processing large-scale spatial data. Sedona extends existing cluster computing systems, such as Apache Spark and Apache Flink, with a set of out-of-the-box distributed Spatial Datasets and Spatial SQL that efficiently load, process, and analyze large-scale spatial data across machines. According to our benchmark and third-party research papers, Sedona runs 2X - 10X faster than other Spark-based geospatial data systems on computation-intensive query workloads. ... -
8
Nuclio
High-Performance Serverless event and data processing platform
Nuclio is an open source and managed serverless platform used to minimize development and maintenance overhead and automate the deployment of data-science-based applications. Real-time performance running up to 400,000 function invocations per second. Portable across low laptops, edge, on-prem and multi-cloud deployments. The first serverless platform supporting GPUs for optimized utilization and sharing. Automated deployment to production in a few clicks from Jupyter notebook. Deploy one of... -
9
RuVector
Self-Learning, Vector Graph Neural Network, and Database built in Rust
RuVector is part of the broader rUv ecosystem of AI engineering tools and focuses on enabling advanced vector-based processing and intelligent system development within agentic and AI-driven pipelines. The project fits into a larger vision of modular, composable AI infrastructure designed to support autonomous agents, data retrieval, and intelligent automation workflows. It emphasizes extensibility and interoperability with modern AI stacks, allowing developers to integrate vector operations into search, reasoning, or generative systems. ... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
Watermill
Building event-driven applications the easy way in Go
Go library for building event-driven applications. Our goal was to create a tool that is easy to understand, even by junior developers. It doesn't matter if you want to do Event-driven architecture, CQRS, Event Sourcing or just stream MySQL Binlog to Kafka. Watermill was designed to process hundreds of thousands of messages per second. Every component is built in a way that allows you to configure it for your needs. You can also implement your own middleware for the router. Watermill is... -
11
Tesla
The flexible HTTP client library for Elixir
The flexible HTTP client library for Elixir, with support for middleware and multiple adapters. Tesla is an HTTP client loosely based on Faraday. It embraces the concept of middleware when processing the request/response cycle. Define module with use Tesla and choose from a variety of middleware. Tesla is built around the concept of composable middlewares. This is very similar to how Plug Router works. All HTTP functions, such as Tesla.get/3 and Tesla.post/4, can take a dynamic client as the... -
12
Kornia
Open Source Differentiable Computer Vision Library
Kornia is a differentiable computer vision library for PyTorch. It consists of a set of routines and differentiable modules to solve generic computer vision problems. At its core, the package uses PyTorch as its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions. Inspired by existing packages, this library is composed by a subset of packages containing operators that can be inserted within... -
13
Hacks
A collection of hacks and one-off scripts
Hacks is a collection of experimental scripts, utilities, and one-off tools created to solve specific problems in security research, data processing, and automation. Rather than being a single cohesive application, it serves as a repository of practical command-line tools that can be used independently or combined into workflows. The scripts cover a wide range of tasks, including URL manipulation, parameter replacement, data extraction, and reconnaissance automation. ... -
14
PHP Code Coverage
Collection, processing, and rendering functionality for PHP code
The php-code-coverage library, authored by Sebastian Bergmann, enables collection, processing, and rendering of PHP code coverage data. It integrates with PHPUnit or other testing frameworks to track which lines, methods, or classes are executed during tests. The library supports generating detailed reports in formats like HTML, Clover, or XML, helping teams understand test completeness and identify untested code paths. -
15
Apache Flink
Stream processing framework with powerful stream
Apache Flink is a distributed engine for stateful computations over data streams and batches, designed for low-latency processing at scale. Its core runtime executes dataflow graphs with fine-grained backpressure and checkpointing, allowing applications to recover consistently from failures. Flink’s event-time model and watermarks enable accurate out-of-order processing, windowing, and complex time semantics that typical real-time systems struggle with. -
16
julep
A new DSL and server for AI agents and multi-step tasks
Julep is a platform for creating AI agents that remember past interactions and can perform complex tasks. It offers long-term memory and manages multi-step processes. Julep enables the creation of multi-step tasks incorporating decision-making, loops, parallel processing, and integration with numerous external tools and APIs. While many AI applications are limited to simple, linear chains of prompts and API calls with minimal branching, Julep is built to handle more complex scenarios. -
17
Smallpond
A lightweight data processing framework built on DuckDB and 3FS
smallpond is a lightweight distributed data processing framework built by DeepSeek, designed to scale DuckDB workloads over clusters using their 3FS (Fire-Flyer File System) backend. The idea is to preserve DuckDB’s fast analytics engine but lift it from single-node to multi-node settings, giving you the ability to operate on large datasets (e.g. petabyte scale) without moving to a heavyweight system like Spark. -
18
ffmpeg.wasm
FFmpeg for browser, powered by WebAssembly
ffmpeg.wasm is a pure WebAssembly (and JavaScript/TypeScript) port of FFmpeg that enables in-browser media recording, conversion, and streaming—letting developers perform video/audio processing entirely client-side without server uploads. Transpiled via Emscripten from FFmpeg and its codecs into WebAssembly. Supports both single-threaded and multi-threaded cores using web workers. Written in TypeScript for improved developer experience. -
19
MathPHP
Powerful modern math library for PHP
Math PHP is a library that brings advanced mathematical functions and data analysis capabilities to PHP applications. It covers a wide range of topics, including linear algebra, calculus, statistics, probability, and numerical analysis. Math PHP is designed for developers and data scientists who require precise and efficient mathematical computations in PHP, making it suitable for scientific computing and data processing. -
20
baoyu-skills
Skills shared by Baoyu for improving daily work efficiency with Claude
baoyu-skills is a comprehensive plugin-based toolkit designed to enhance productivity within AI-assisted development environments, particularly Claude Code, by providing a wide range of modular “skills” for content generation, automation, and data processing. The project organizes its functionality into categories such as content creation, AI generation, and utility tools, enabling users to extend their workflows through reusable components. Each skill is implemented as a structured module with its own configuration, scripts, and execution logic, allowing for flexible customization and extension. ... -
21
Apache InLong
Apache InLong - a one-stop integration framework for massive data
Apache InLong is a one-stop integration framework for massive data that provides automatic, secure and reliable data transmission capabilities. InLong supports both batch and stream data processing at the same time, which offers great power to build data analysis, modeling and other real-time applications based on streaming data. InLong (应龙) is a divine beast in Chinese mythology who guides the river into the sea, and it is regarded as a metaphor of the InLong system for reporting data streams. ... -
22
Kotlin Dataframe
Structured data processing in Kotlin
Data frame is an abstraction for working with structured data. Essentially it’s a 2-dimensional table with labeled columns of potentially different types. You can think of it like a spreadsheet or SQL table, or a dictionary of series objects. The handiness of this abstraction is not in the table itself but in a set of operations defined on it. The Kotlin Dataframe library is an idiomatic Kotlin DSL defining such operations. The process of working with data frame is often called data... -
23
Thulite
Web framework designed for speed, security, and SEO
Thulite is an AI-powered search and recommendation engine that enhances search functionality in applications. It provides intelligent query processing, result ranking, and personalized recommendations. -
24
Bacalhau
Community-driven, simple, yet powerful framework
Bacalhau is a decentralized compute platform for running jobs on data stored across distributed networks, like IPFS or Filecoin, without moving the data to centralized cloud environments. It allows developers to run containerized workloads close to where the data lives, reducing latency, cost, and privacy risks. Bacalhau supports various runtime environments and is designed to make decentralized data processing as accessible as traditional cloud computing. ... -
25
Lexbor
Lexbor is development of an open source HTML Renderer library
Lexbor is the development of a web browser engine available as a software library; it ships with a free license and has no extra dependencies. For us, speed is an absolute must-have. In our development process, we focus on fastest parsing techniques for HTML, CSS, and fonts, fastest data processing methods, and fastest ways to serve content to end users. Whether you are building a backend that handles millions of HTML documents or a UI-heavy user app, your software’s response rate always matters to users and developers alike. Lexbor’s code is optimized for ease of access in end-user applications and across programming languages. ...
