Search Results for "data processing" - Page 4
Sort By:
Showing 418 open source projects for "data processing"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
1
Adafruit Unified Sensor Driver
Common sensor library
Many small embedded systems exist to collect data from sensors, analyze the data, and either take appropriate action or send that sensor data to another system for processing. One of the many challenges of embedded systems design is the fact that parts you used today may be out of production tomorrow, or system requirements may change and you may need to choose a different sensor down the road. -
2
Google Cloud Dataflow Template Pipelines
Cloud Dataflow Google-provided templates for solving data tasks
DataflowTemplates is the source repository for Google-provided Dataflow templates that are intended to solve large-scale in-cloud data processing tasks without requiring users to build everything from scratch in a full development environment. The repository is centered on templated pipelines powered by Google Cloud Dataflow and Apache Beam, making it easier to run common integration and movement jobs such as data import, export, backup, restore, and bulk API operations. ... -
3
Sov.ai
A curated list of practical financial machine learning tools and apps
Financial Machine Learning is a curated directory of practical tools, repositories, datasets, papers, and educational resources for quantitative finance. It organizes material across trading, forecasting, portfolio construction, risk, alternative data, and financial machine learning techniques. Dedicated sections cover supervised and unsupervised learning, deep learning, reinforcement learning, natural language processing, and time-series analysis. Entries include descriptions, popularity data, maintenance indicators, and editorial ratings to help readers compare resources. The main README highlights top-ranked items, while the wiki contains the larger catalogue. ... -
4
JC
CLI tool and python library
...This allows piping of output to tools like jq and simplifying automation scripts. jc JSONifies the output of many CLI tools and file types for easier parsing in scripts. This allows further command-line processing of output with tools like jq or jello by piping commands. The JC parsers can also be used as python modules. In this case, the output will be a python dictionary, or a list of dictionaries, instead of JSON. Two representations of the data are available. The default representation uses a strict schema per parser and converts known numbers to int/float JSON values. ... -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
5
YoutubeExplode
Abstraction layer over YouTube's internal API
YoutubeExplode is a .NET library that provides a high-level abstraction for interacting with YouTube data, enabling developers to retrieve metadata and download media streams programmatically. The project exposes a clean API that allows applications to query videos, playlists, channels, and search results without relying on the official YouTube Data API. Under the hood, the library parses raw page data and leverages reverse-engineered internal endpoints to obtain structured information and stream manifests. ... -
6
XLSX
Go (golang) library for reading and writing XLSX files
...These can be used to modify the behavior of the resultant struct, in particular they replace the `…WithRowLimit` variants of those methods with the result of calling `xlsx.RowLimit` and they add the ability to define a custom backing store for the spreadsheet data to be held in whilst processing. The full API docs can be viewed using go’s built in documentation tool. -
7
LoopVectorization.jl
Macro(s) for vectorizing loops
LoopVectorization.jl is a Julia package for accelerating numerical loops by automatically applying SIMD (Single Instruction, Multiple Data) vectorization and other low-level optimizations. It analyzes loops and generates highly efficient code that leverages CPU vector instructions, making it ideal for performance-critical computing in fields such as scientific computing, signal processing, and machine learning. -
8
XZ Utils
Open-source compression utility and library
xz is a widely used open-source compression utility and library that implements the high-ratio LZMA and LZMA2 compression algorithms. It provides both command-line tools and a reusable C library, enabling developers and system administrators to compress and decompress files efficiently across many environments. The project is known for delivering strong compression performance while maintaining reasonable memory usage, making it suitable for software distribution, backups, and archival... -
9
Trame
Weave various components and technologies into a Web App
...It enables the integration of various components and technologies, such as VTK and ParaView, into web applications written entirely in Python. With best-in-class platforms at its core, trame provides complete control of 3D visualizations and data processing. Developers benefit from a write-once environment from trame. trame is an open source project licensed under Apache License Version 2.0 which allows users to create open source or commercial applications without any licensing worries. By relying simply on Python and HTML, trame focuses on one's data and associated analysis and visualizations while hiding the complications of web development. -
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
10
Stanza
Stanford NLP Python library for many human languages
Stanza is a collection of accurate and efficient tools for the linguistic analysis of many human languages. Starting from raw text to syntactic analysis and entity recognition, Stanza brings state-of-the-art NLP models to languages of your choosing. Stanza is a Python natural language analysis package. It contains tools, which can be used in a pipeline, to convert a string containing human language text into lists of sentences and words, to generate base forms of those words, their parts of... -
11
FasterLIO
Faster-LIO: Lightweight Tightly Coupled Lidar-inertial Odometry
...It builds on FastLIO2 while replacing key spatial operations with parallel sparse incremental voxels. The design targets high update rates for both solid-state and spinning LiDAR sensors. It fuses LiDAR measurements with inertial data through an iterative Kalman-filter-based estimation pipeline. The software supports online ROS operation as well as offline processing of recorded bag files. Users can compile it with CMake or catkin, run it through Docker, visualize output in RViz, and save trajectories and point clouds. Configuration files support multiple sensor datasets, while selectable voxel implementations allow performance and stability tuning. -
12
GH Archive
GH Archive is a project to record the public GitHub timeline
GH Archive collects public GitHub activity and preserves it as a historical dataset for analysis. It polls GitHub’s event feed and groups the raw responses into compressed hourly JSON archives. The records cover actions such as commits, forks, issue activity, comments, and repository membership changes. Researchers can download specific hours, days, or longer periods with ordinary HTTP tools and process the events in their own systems. The dataset is also published through Google BigQuery... -
13
mosdepth
fast BAM/CRAM depth calculation for WGS, exome, or targeted sequencing
mosdepth is a fast BAM/CRAM depth calculation tool for genomic data, allowing efficient computation of sequencing coverage. -
14
Async PHP
Easily run code asynchronously
...It helps optimize performance by executing long-running or resource-intensive tasks concurrently, instead of sequentially. The library is easy to use and integrates well with existing PHP applications, making it suitable for batch processing, data scraping, or any scenario where concurrency can boost efficiency. -
15
Zigbee Herdsman Converters
Collection of device converters to be used with zigbee-herdsman
...The repository includes testing, formatting, build tooling, external-definition support, post-processing, and OTA-related interfaces for maintaining a large evolving device catalog. -
16
RuleGo
Component orchestration rule engine framework for Go
...It’s lightweight, embeddable, orchestration-ready, and built for flexible composition of business logic into reusable components. No external middleware dependencies, efficient data processing and linkage on low-cost devices, suitable for IoT edge computing. Embedded and Standalone Deployment modes. Supports embedding RuleGo into existing applications. It can also be deployed independently as middleware, providing rule engine and orchestration services. -
17
The Algorithms Python
All Algorithms implemented in Python
The Algorithms-Python project is a comprehensive collection of Python implementations for a wide range of algorithms and data structures. It serves primarily as an educational resource for learners and developers who want to understand how algorithms work under the hood. Each implementation is designed with clarity in mind, favoring readability and comprehension over performance optimization. The project covers various domains including mathematics, cryptography, machine learning, sorting,... -
18
kube-fledged
Kubernetes operator to create and manage cache of container images
kube-fledged is a kubernetes operator for creating and managing a cache of container images directly on the worker nodes of a Kubernetes cluster. It allows a user to define a list of images and onto which worker nodes those images should be cached (i.e. pulled). As a result, application pods start almost instantly, since the images need not be pulled from the registry. kube-fledged provides CRUD APIs to manage the lifecycle of the image cache, and supports several configurable parameters to... -
19
KubeEdge
Kubernetes Native Edge Computing Framework (project under CNCF)
...It also supports MQTT which enables edge devices to access through edge nodes. With KubeEdge it is easy to get and deploy existing complicated machine learning, image recognition, event processing, and other high-level applications to the Edge. With business logic running at the Edge, much larger volumes of data can be secured & processed locally where the data is produced. With data processed at the Edge, the responsiveness is increased dramatically and data privacy is protected. -
20
Halide
A language for fast, portable data-parallel computation
Halide is a programming language for fast, portable data-parallel computation. It was designed to make writing high-performance image and array processing code much easier on modern machines. It works on all major operating systems and with several CPU architectures (X86, ARM, MIPS, Hexagon, PowerPC) and GPU Compute APIs (CUDA, OpenCL, OpenGL, among others). It isn't a standalone programming language however; rather it is embedded in C++ which means that you write C++ code, building an in-memory representation of a Halide pipeline using Halide's C++ API. ... -
21
Absinthe
The GraphQL toolkit for Elixir
...If you're new to GraphQL, we suggest you read up a bit on GraphQL's foundational principles before you dive into Absinthe. Absinthe's functionality generally falls into two broad areas. Defines the structure of data entities and the relationships between, as well as the available queries, mutations, and subscriptions, using an elegant collection of declarative macros. Absinthe schemas are defined using easy-to-read macros that build and verify their structure at compile-time, preventing runtime errors and increasing performance. The entire query processing pipeline is configurable. ... -
22
LaReview
The code review workbench
...Instead of overwhelming developers with raw diffs or automated comment spam, the tool analyzes code changes and generates an intent-driven review plan that groups changes into logical flows such as authentication, API behavior, or data handling, and prioritizes them based on risk. It operates as a desktop application with CLI integration, allowing users to launch reviews directly from their terminal while keeping all processing local to ensure security and prevent data leakage. The system presents reviews as hierarchical task trees, enabling developers to work through changes step by step, attach notes, and track progress across different review concerns. -
23
AssemblyScript
A TypeScript-like language for WebAssembly
...AssemblyScript integrates with the WebAssembly ecosystem, enabling seamless interaction with JavaScript through standard APIs. It is widely used in scenarios where performance and portability are essential, such as gaming, data processing, and edge computing. -
24
go1pylib
go1pylib is a Python library designed to control the Go1 robot
go1pylib is a Python library designed to control the Go1 robot by Unitree Robotics. It provides an easy-to-use interface for robot movement, state management, collision avoidance, battery monitoring, and MQTT communication. Ideal for research and robotics development. -
25
TensorFlow
TensorFlow is an open source library for machine learning
Originally developed by Google for internal use, TensorFlow is an open source platform for machine learning. Available across all common operating systems (desktop, server and mobile), TensorFlow provides stable APIs for Python and C as well as APIs that are not guaranteed to be backwards compatible or are 3rd party for a variety of other languages. The platform can be easily deployed on multiple CPUs, GPUs and Google's proprietary chip, the tensor processing unit (TPU). TensorFlow...
