Search Results for "data processing"
Sort By:
Showing 188 open source projects for "data processing"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
1
jq
Lightweight and flexible command-line JSON processor
jq is like sed for JSON data - you can use it to slice, filter, map and transform structured data with the same ease that sed, awk, grep and friends let you play with text. jq is written in portable C, and it has zero runtime dependencies. You can download a single binary, scp it to a far away machine of the same type, and expect it to work. jq can mangle the data format that you have into the one that you want with very little effort, and the program to do so is often shorter and simpler... -
2
Databend
Cloud-native open source data warehouse for analytics and AI queries
Databend is an open source cloud-native data warehouse designed for large-scale analytics and modern data workloads. Built in Rust, the system focuses on high performance, scalability, and efficient data processing for analytical queries. It is designed with a separation of compute and storage, allowing compute nodes to scale independently while storing data in object storage systems. -
3
HDF5
Official HDF5® Library Repository
HDF5 (Hierarchical Data Format v5) is a widely-used data management library and file format for storing large and complex scientific data sets efficiently. -
4
fluentbit
Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX
Fluent Bit is a super-fast, lightweight, and highly scalable logging and metrics processor and forwarder. It is the preferred choice for cloud and containerized environments. A robust, lightweight, and portable architecture for high throughput with low CPU and memory usage from any data source to any destination. Proven across distributed cloud and container environments. Highly available with I/O handlers to store data for disaster recovery. Granular management of data parsing and routing.... -
Build Securely on Azure with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
collapse
Advanced and Fast Data Transformation in R
collapse is a high-performance R package designed for fast and efficient data transformation, aggregation, reshaping, and statistical computation. Built to offer a more performant alternative to dplyr and data.table, it is particularly well-suited for large datasets and econometric applications. It operates on base R data structures like data frames and vectors and uses highly optimized C++ code under the hood to deliver significant speed improvements. collapse also includes tools for... -
6
Performance Co-Pilot (PCP)
Performance Co-Pilot
Performance Co-Pilot (PCP) provides a framework and services to support system-level performance monitoring and management. It presents a unifying abstraction for all of the performance data in a system, and many tools for interrogating, retrieving and processing that data. PCP is a feature-rich, mature, extensible, cross-platform toolkit supporting both live and retrospective analysis. The distributed PCP architecture makes it especially useful for those seeking centralized monitoring of distributed processing. -
7
TDengine
Open-source time-series database with high-performance and scalability
Enables efficient, real-time data ingestion, processing and monitoring of TB and even PB scale data per day, generated by billions of sensors and data collectors. TDengine can be widely applied to IoT, Industrial Internet, Connected Vehicles, DevOps, Energy , Finance and many other use-cases. TDengine’s innovative design and purpose-built storage engine outperforms other time-series databases for data ingestion, querying and data compression while significantly reducing storage and computing costs. ... -
8
WebP Codec
Library to encode and decode images in WebP format
libwebp is the reference codec library for Google’s WebP image format, providing both encoding and decoding along with command-line tools. It supplies cwebp to compress images into WebP and dwebp to decompress them back, making it easy to test quality/size trade-offs across presets and tuning parameters. The GitHub repository is a mirror; the canonical source of truth lives on Chromium’s git, and developer docs are hosted on WebP’s portal. The project underpins WebP support across browsers,... -
9
Lexbor
Lexbor is development of an open source HTML Renderer library
Lexbor is the development of a web browser engine available as a software library; it ships with a free license and has no extra dependencies. For us, speed is an absolute must-have. In our development process, we focus on fastest parsing techniques for HTML, CSS, and fonts, fastest data processing methods, and fastest ways to serve content to end users. Whether you are building a backend that handles millions of HTML documents or a UI-heavy user app, your software’s response rate always matters to users and developers alike. Lexbor’s code is optimized for ease of access in end-user applications and across programming languages. ... -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
10
Acl
A powerful server and network library, including coroutine
The Acl (Advanced C/C++ Library) project a is powerful multi-platform network communication library and service framework, supporting LINUX, WIN32, Solaris, FreeBSD, MacOS, AndroidOS, iOS. Many applications written by Acl run on these devices with Linux, Windows, iPhone and Android and serve billions of users. There are some important modules in Acl project, including network communcation, server framework, application protocols, multiple coders, etc. The common protocols such as... -
11
ffmpeg.wasm
FFmpeg for browser, powered by WebAssembly
ffmpeg.wasm is a pure WebAssembly (and JavaScript/TypeScript) port of FFmpeg that enables in-browser media recording, conversion, and streaming—letting developers perform video/audio processing entirely client-side without server uploads. Transpiled via Emscripten from FFmpeg and its codecs into WebAssembly. Supports both single-threaded and multi-threaded cores using web workers. Written in TypeScript for improved developer experience. -
12
syslog-ng
Log management solution that improves the performance of SIEM
syslog-ng is the log management solution that improves the performance of your SIEM solution by reducing the amount and improving the quality of data feeding your SIEM. With syslog-ng Store Box, you can find the answer. Search billions of logs in seconds using full text queries with Boolean operators to pinpoint critical logs. syslog-ng Store Box provides secure, tamper-proof storage and custom reporting to demonstrate compliance. syslog-ng can deliver data from a wide variety of sources to... -
13
Format Converter - Intel HEX, Elf32, BIN
File Format Converter Between HEX / BIN / ELF
Intel HEX File Processing Software File Format Converter Between HEX / BIN / ELF The source code was created and debugged in the eS development environment (for Linux and Windows) and is an example of its use. https://esolution.sourceforge.io/ -
14
XZ Utils
Open-source compression utility and library
xz is a widely used open-source compression utility and library that implements the high-ratio LZMA and LZMA2 compression algorithms. It provides both command-line tools and a reusable C library, enabling developers and system administrators to compress and decompress files efficiently across many environments. The project is known for delivering strong compression performance while maintaining reasonable memory usage, making it suitable for software distribution, backups, and archival... -
15
pg_textsearch
PostgreSQL extension for BM25 relevance-ranked full-text search
The pg_textsearch project is a PostgreSQL extension designed to enhance full-text search capabilities within the database environment. It provides tools and optimizations for indexing and querying textual data efficiently, enabling developers to build search functionality directly into their applications without relying on external search engines. The extension integrates with PostgreSQL’s native indexing mechanisms, allowing for scalable and performant search operations across large datasets. It also supports advanced query features such as ranking, filtering, and linguistic processing. ... -
16
The RAD Debugger Project
A native, user-mode, multi-process, graphical debugger
...The debugger is user-mode and can work with multiple processes during a debugging session. PDB information is converted into the project’s custom RAD Debug Info format for faster and more controlled processing. The repository also contains radbin, utilities for inspecting and converting binary information. A related RAD Linker targets large x64 PE/COFF programs and can produce either standard PDB or native RDI data. The project remains in alpha while prioritizing reliability, performance, stepping, evaluation, process control, and large-project compile-debug workflows. -
17
Gwyddion
Scanning probe microscopy data visualisation and analysis
A data visualization and processing tool for scanning probe microscopy (SPM, i.e. AFM, STM, MFM, SNOM/NSOM, ...) and profilometry data, useful also for general image and 2D data analysis. -
18
miniaudio
Audio playback and capture library written in C,
miniaudio is written in C with no dependencies except the standard library and should compile cleanly on all major compilers without the need to install any additional development packages. All major desktop and mobile platforms are supported. miniaudio gives you complete flexibility. With the low-level API, just initialize a connection to the device and send or receive raw audio data. The modular design of miniaudio allows you to use the low-level API without compromising your ability to... -
19
BibDesk
Bibliography manager for Mac OS X
BibDesk is a graphical bibTeX bibliography manager for Mac OS X. -
20
Octave Forge
A collection of packages providing extra functionality for GNU Octave
Octave Forge is a central location for collaborative development of packages for GNU Octave. The Octave Forge packages expand Octave's core functionality by providing field specific features via Octave's package system. See https://octave.sourceforge.io/packages.php for a list of all available packages. GNU Octave is a high-level interpreted language, primarily intended for numerical computations. It provides capabilities for the numerical solution of linear and nonlinear problems, and... -
21
HEALPix
Data Analysis, Simulations and Visualization on the Sphere
Software for pixelization, hierarchical indexation, synthesis, analysis, and visualization of data on the sphere. Please acknowledge HEALPix by quoting the web page http://healpix.sourceforge.net (or https://healpix.sourceforge.io) and publication: K.M. Gorski et al., 2005, Ap.J., 622, p.759 Full software documentation available at https://healpix.sourceforge.io/documentation.php Wiki Pages: https://sourceforge.net/p/healpix/wiki/Home Exchanging Data with HEALPix (in FITS files):... -
22
Madagascar is a software package for geophysical data processing and reproducible numerical experiments. The package mission is to provide a convenient environment for researchers working with digital image and data processing in geophysics and related fields.
-
23
libsombrero
Astronomical object/structure detection from 1D and 2D data sets.
Sombrero is a fast wavelet image processing and object detection C library for astronomical images. Sombrero is named after the "Mexican Hat" shape of the wavelet masks used in image convolution and is released under the GNU LGPL library. -
24
Teem
Tools to process and visualize scientific data and images
Teem is a set of C libraries for manipulation, measurement, and visualization of structured scientific data. Includes N-dimensional image I/O and processing, volume rendering, diffusion tensor processing, and more. Latest code via git clone http://git.code.sf.net/p/teem/teem.git . Join us at https://discord.gg/xBBqZGXkF7 -
25
BioXTAS RAW
Processing and analysis of Small Angle X-ray Scattering (SAXS) data.
BioXTAS RAW is a program for analysis of Small-Angle X-ray Scattering (SAXS) data. The software enables: creation of 1D scattering profiles from 2D detector images, standard data operations such as averaging and subtraction, analysis of radius of gyration (Rg) and molecular weight, and advanced analysis using GNOM and DAMMIF as well as electron density reconstructions using DENSS. It also allows easy processing of inline SEC-SAXS data and data deconvolution using the evolving factor analysis (EFA) or the regularized alternating least squares (REGALS) methods. ...
