Search Results for "data flow"
Sort By:
Showing 57 open source projects for "data flow"
View related business solutions-
Go from Data Warehouse to Data and AI platform with BigQueryBigQuery is more than a data warehouse—it's an autonomous data-to-AI platform. Use familiar SQL to train ML models, run time-series forecasts, and generate AI-powered insights with native Gemini integration. Built-in agents handle data engineering and data science workflows automatically. Get $300 in free credit, query 1 TB, and store 10 GB free monthly.
-
AI-powered service management for IT and enterprise teamsGive your IT, operations, and business teams the ability to deliver exceptional services—without the complexity. Maximize operational efficiency with refreshingly simple, AI-powered Freshservice.
-
1
go-streams
A lightweight stream processing library for Go
A lightweight stream processing library for Go. go-streams provides a simple and concise DSL to build data pipelines. In computing, a pipeline, also known as a data pipeline, is a set of data processing elements connected in series, where the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in time-sliced fashion. Some amount of buffer storage is often inserted between elements. -
2
PowerSystems.jl
Data structures in Julia to enable power systems analysis
The PowerSystems.jl package provides a rigorous data model using Julia structures to enable power systems analysis and modeling. In addition to stand-alone system analysis tools and data model building, the PowerSystems.jl package is used as the foundational data container for the PowerSimulations.jl and PowerSimulationsDynamics.jl packages. PowerSystems.jl supports a limited number of data file formats for parsing. -
3
GoJS
JavaScript diagramming library for interactive flowcharts & org charts
Build interactive flowcharts or flow diagrams. Let your users build, modify, and save diagrams with JSON model output. Visualize state charts and other behavior diagrams. Create diagrams with live updates to monitor state, or interactive diagrams for planning. GoJS allows considerable customization of links and nodes to build all kinds of diagrams. Visualize flow, or connect pipes. -
4
Rocket.jl
Functional reactive programming extensions library for Julia
Rocket.jl is a Julia package for reactive programming using Observables, to make it easier to work with asynchronous data. Rocket.jl has been designed with a focus on performance and modularity. -
$300 in Free Credit for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credit—no hidden charges. Test, build, and deploy without risk. Use your credit across the Google Cloud platform to find what works best for your needs. After your credits are used, continue building with free monthly usage products. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
5
YAO
Yao A low code engine to create web services and dashboard
...Yao is a low-code engine that creates a database model, writes API services, and describes dashboard interface just by JSON for web & hardware, no code, and 10x productivity. Yao is based on the flow-based programming idea, developed in the Go language, and supports multiple ways to expand the data stream processor. This makes Yao extremely versatile, which can replace programming languages in most scenarios, and is 10 times more efficient than traditional programming languages in terms of reusability and coding efficiency; application performance and resource ratio Better than PHP, JAVA and other languages. ... -
6
Circuitscape.jl
Algorithms from circuit theory to predict connectivity
Circuitscape is an open-source program that uses circuit theory to model connectivity in heterogeneous landscapes. Its most common applications include modeling the movement and gene flow of plants and animals, as well as identifying areas important for connectivity conservation. The new Circuitscape is built entirely in the Julia language, a new programming language for technical computing. Julia is built from the ground up to be fast. As such, this offers a number of advantages over the... -
7
ReverseDiff
Reverse Mode Automatic Differentiation for Julia
ReverseDiff is a fast and compile-able tape-based reverse mode automatic differentiation (AD) that implements methods to take gradients, Jacobians, Hessians, and higher-order derivatives of native Julia functions (or any callable object, really). While performance can vary depending on the functions you evaluate, the algorithms implemented by ReverseDiff generally outperform non-AD algorithms in both speed and accuracy. -
8
GenStage
Producer and consumer actors with back-pressure for Elixir
GenStage is a specification and set of behaviours for building demand-driven data pipelines on the BEAM. It formalizes the roles of producers, consumers, and producer-consumers, using back-pressure so that fast producers don’t overwhelm downstream stages. Developers implement callbacks like handle_demand and handle_events to control how items are emitted, transformed, and consumed across asynchronous boundaries. -
9
AWS SDK for pandas
Easy integration with Athena, Glue, Redshift, Timestream, Neptune
aws-sdk-pandas (formerly AWS Data Wrangler) bridges pandas with the AWS analytics stack so DataFrames flow seamlessly to and from cloud services. With a few lines of code, you can read from and write to Amazon S3 in Parquet/CSV/JSON/ORC, register tables in the AWS Glue Data Catalog, and query with Amazon Athena directly into pandas. The library abstracts efficient patterns like partitioning, compression, and vectorized I/O so you get performant data lake operations without hand-rolling boilerplate. ... -
Easily Host LLMs and Web Apps on Cloud RunRun frontend and backend services, batch jobs, host LLMs, and queue processing workloads without the need to manage infrastructure. Cloud Run gives you on-demand GPU access for hosting LLMs and running real-time AI—with 5-second cold starts and automatic scale-to-zero so you only pay for actual usage. New customers get $300 in free credit to start.
-
10
Superstruct
A simple and composable way to validate data in JavaScript
This project is a lightweight validation library for JavaScript and TypeScript that helps you define data “shapes” and validate runtime values against them. Instead of relying only on compile-time typing, it focuses on the reality that many apps receive unknown input from APIs, forms, and external systems, and those values need runtime checks. Its API is intentionally familiar if you’ve used TypeScript, Flow, Go structs, or GraphQL schemas, but the output is oriented around runtime correctness. ... -
11
Acl
A powerful server and network library, including coroutine
The Acl (Advanced C/C++ Library) project a is powerful multi-platform network communication library and service framework, supporting LINUX, WIN32, Solaris, FreeBSD, MacOS, AndroidOS, iOS. Many applications written by Acl run on these devices with Linux, Windows, iPhone and Android and serve billions of users. There are some important modules in Acl project, including network communcation, server framework, application protocols, multiple coders, etc. The common protocols such as... -
12
FlowViewer
FlowViewer is a web-based netflow data analysis tool.
FlowViewer provides a convenient web-based user interface to Mark Fullmer’s flow-tools suite and CMU's netflow data capture/analyzer, SiLK. The inclusion of the underlying SiLK tool set enables FlowViewer users to continue to use the tool with the newer IPFIX netflow data protocol, which includes support for IPv6 and Cisco's v9 and FNF netflow. FlowViewer has been developed for NASA’s Earth Sciences Data and Information System (ESDIS) networks, and credit goes to NASA for their usual outstanding support of innovation. ... -
13
elder-pt
data flow framework for data analysis applications
Data analysis in nuclear/particle physics experiments is based on data processing of event-by-event data. Each event contains information measured by arrays of detectors. This data has to be combined, processed, interpreted and reduced in size until the physics information of interest is revealed. Eventually, parts of the data is visualized by sorting it into histograms. Elder-Pt is the prototype of a framework, that facilitates the task of event-by-event data analysis. It provides: 1) a... -
14
...It implements the event stream processing as a library embeddable in C++ and Perl. You can think of the Complex Event Processing engine as an in-memory database driven by triggers, or a data-flow machine, or a spreadsheet on steroids (and without the GUI part).
-
15
TurboVNC
High-speed, 3D-friendly, TightVNC-compatible remote desktop software
TurboVNC is a high-performance, enterprise-quality version of VNC based on TightVNC, TigerVNC, and X.org. It contains a variant of Tight encoding that is tuned for maximum performance and compression with 3D applications (VirtualGL), video, and other image-intensive workloads. TurboVNC, in combination with VirtualGL, provides a complete solution for remotely displaying 3D applications with interactive performance. TurboVNC's high-speed encoding methods have been adopted by TigerVNC and... -
16
GETL
ETL engine based on Groovy
P.S. Dear friends. Repository migration to https://github.com/ascrus/getl . You can download jar file from this site or maven. GETL - based package in Groovy, which automates the work of loading and transforming data. His name is an acronym for «Groovy ETL». GETL is a set of libraries of pre-built classes and objects that can be used to solve problems unpacking, transform and load data into programs written in Groovy, or Java, as well as from any software that supports the work with... -
17
ProxSDP.jl
Semidefinite programming optimization solver
ProxSDP is an open-source semidefinite programming (SDP) solver based on the paper "Exploiting Low-Rank Structure in Semidefinite Programming by Approximate Operator Splitting". The main advantage of ProxSDP over other state-of-the-art solvers is the ability to exploit the low-rank structure inherent to several SDP problems. -
18
BitSail
BitSail is a distributed high-performance data integration engine
BitSail is ByteDance's open source data integration engine which is based on distributed architecture and provides high performance. It supports data synchronization between multiple heterogeneous data sources, and provides global data integration solutions in batch, streaming, and incremental scenarios. At present, it serves almost all business lines in ByteDance, such as Douyin, Toutiao, etc., and synchronizes hundreds of trillions of data every day. BitSail has been widely used and... -
19
SQLBucket
Lightweight library to write, orchestrate and test your SQL ETL
SQLBucket is a lightweight framework to help write, orchestrate and validate SQL data pipelines. It gives the possibility to set variables and introduces some control flow using the fantastic Jinja2 library. It also implements a very simplistic unit and integration test framework where you can validate the results of your ETL in the form of SQL checks. With SQLBucket, you can apply TDD principles when writing data pipelines. -
20
Transporter
Sync data between persistence engines, like ETL only not stodgy
...Support for other DBs will be added later on. Other adaptors may or may not work. You're encouraged to still use v0.5.2 for non mongo/postgres migrations. Transporter allows the user to configure a number of data adaptors as sources or sinks. These can be databases, files or other resources. Data is read from the sources, converted into a message format, and then send down to the sink where the message is converted into a writable format for its destination. The user can also create data transformations in JavaScript which can sit between the source and sink and manipulate or filter the message flow. ... -
21
Perceptron
The birth of modern video feedback art.
Perceptron is a video feedback engine with a variety of extraordinary graphical effects. Perceptron is an endless flow of transforming visuals. Perceptron * recursively transforms images and video streams in realtime and produces a combination of Julia fractals, IFS fractals, and chaotic patterns due to video feedback * evolves geometric patterns into the realm of infinite details and deepens the thought * records animations (movies) * saves and opens presets... -
22
RDS - JS - Examples
TypeScript/JavaScript example code using the RDS API
Rich Data Services (or RDS) is a suite of REST APIs designed by Metadata Technology North America (MTNA) to meet various needs for data engineers, managers, custodians, and consumers. RDS provides a range of services including data profiling, mapping, transformation, validation, ingestion, and dissemination. For more information about each of these APIs and how you can incorporate or consume them as part of your work flow please visit the MTNA website. ... -
23
dycomodel
Dynamic Consumption Modeling
Dynamic Consumption Modeling to create the model of the feature consumption’s flow taking in consideration: the data from prior period of consumption (one time produced) specific model suggested by system and customized by customer everyday/periodically inventory quantity and to predict the remaining quantity of items set the reordering points (first point, with fixed quantity, for fixed dates) GitHub: https://github.com/surban1974/dycomodel -
24
VMS Draw
user-friendly access to the latest computational spectroscopy tools
VMS Draw provides general utilities (e.g. normalization, conversion, and other manipulations of several spectra at the same time) and a flexible graphical user interface (GUI) for an easy use by non-specialists which allows a seamless flow of information between experimentally and theoretically oriented researchers. Finally, it permits effective interactions with other electronic structure codes [e.g., nuclear magnetic resonance (NMR), microwave] even if Gaussian 16 offers the widest set... -
25
SZT-bigdata
SZT‑bigdata is an open source project
SZT‑bigdata is an open-source project analyzing real Shenzhen metro (subway) card usage data using big‑data frameworks like Spark, Hadoop, Hive, Kafka, Flink, ClickHouse, HBase, and Elasticsearch. Aimed at exploring transit passenger flow patterns and system optimization using a variety of Scala-based technologies.
