Search Results for "data collection algorithm"
Sort By:
Showing 160 open source projects for "data collection algorithm"
View related business solutions-
Keep company data safe with Chrome EnterpriseMake AI work your way with Chrome Enterprise. Block unapproved sites and set custom data controls that align with your company's policies.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Zstandard
Zstandard - Fast real-time compression algorithm
Zstandard is a fast compression algorithm, providing high compression ratios. It also offers a special mode for small data, called dictionary compression. The reference library offers a very wide range of speed / compression trade-off, and is backed by an extremely fast decoder (see benchmarks below). Zstandard library is provided as open source software using a BSD license. -
2
Alibaba iLogtail
Fast and Lightweight Observability Data Collector
iLogtail was born for observable scenarios and has many production-level features such as lightweight, high performance, and automated configuration, which are widely used internally by Alibaba Group and tens of thousands of external Alibaba Cloud customers. You can deploy it in physical machines, Kubernetes and other environments to collect telemetry data, such as logs, traces and metrics. Supports a variety of Logs, Traces, and Metrics data collection, and is friendly to container and Kubernetes environment support. The resource cost of data collection is quite low, 5-20 times better than similar telemetry data collection Agent performance. High stability, used in the production of Alibaba and tens of thousands of Alibaba Cloud customers, and collecting dozens of petabytes of observable data every day with nearly tens of millions deployments. -
3
ThingsBoard
Device management, data collection, processing and visualization
ThingsBoard is an open-source IoT platform for data collection, processing, visualization, and device management. It enables device connectivity via industry standard IoT protocols, MQTT, CoAP and HTTP and supports both cloud and on-premises deployments. ThingsBoard combines scalability, fault-tolerance and performance so you will never lose your data. Provision, monitor and control your IoT entities in a secure way using rich server-side APIs. -
4
Zipkin
Distributed tracing system to gather timing data
Zipkin is a distributed tracing system. It helps gather timing data needed to troubleshoot latency problems in service architectures. Features include both the collection and lookup of this data. If you have a trace ID in a log file, you can jump directly to it. Otherwise, you can query based on attributes such as service, operation name, tags and duration. Some interesting data will be summarized for you, such as the percentage of time spent in a service, and whether or not operations failed. ... -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
5
Brotli
Brotli compression format
...This happens when "one-shot" decoding API is used (or input chunk for streaming API is not limited), input size (chunk size) is larger than 2GiB, and input contains uncompressed blocks. After the overflow happens, memcpy is invoked with a gigantic num value, that will likely cause the crash. Brotli is a generic-purpose lossless compression algorithm that compresses data using a combination of a modern variant of the LZ77 algorithm, Huffman coding and 2nd order context modeling, with a compression ratio comparable to the best currently available general-purpose compression methods. It is similar in speed with deflate but offers more dense compression. The specification of the Brotli Compressed Data Format is defined in RFC 7932. ... -
6
LZ4
Extremely fast compression algorithm
LZ4 is lossless compression algorithm, providing compression speed > 500 MB/s per core (>0.15 Bytes/cycle). It features an extremely fast decoder, with speed in multiple GB/s per core (~1 Byte/cycle). A high compression derivative, called LZ4_HC, is available, trading customizable CPU time for compression ratio. LZ4 library is provided as open-source software using a BSD license. -
7
Data Crow
The ultimate cataloguer
Data Crow allows you to use the standard movie & video (divx, xvid, DVD, Blu-ray, etc), book (and eBooks), images, board games, comic books, games & software, music (mp3 and other music files) cataloguing modules. Besides these modules, which you can change to fit your requirements, you can create new modules (want to catalogue your stamps, equipment, or anything else?). The GUI is skinnable. Reporting (using JasperReports and their community edition JasperSoft Developer Studio ), loan... -
8
Loggie
A lightweight, cloud-native data transfer agent and aggregator
Loggie is a lightweight, high-performance, cloud-native agent and aggregator based on Golang. Loggie includes LogConfig/ClusterLogConfig/Interceptor/Sink CRDs, allowing for the creation of data collection, transfer, processing, and sending pipelines through simple YAML file creation. Supports deployment as an independent intermediate machine, which can receive aggregated data sent by Loggie Agent and can also be used to consume and process various data sources. Configure Filebeat and Loggie to collect logs, and send them to a Kafka topic without using client compression, with the Kafka topic partition configured as 3. ... -
9
Netdisco
A web-based network management tool
Netdisco is a web-based network management tool suitable for small to very large networks. It collects IP and MAC address data into a PostgreSQL database using SNMP, CLI, or device APIs. -
Connect every part of your business to one bank accountNorth One is proudly built for small businesses, startups and freelancers across America. Make payments easily, keep tabs on your money and put your finances on autopilot through smart integrations with the tools you’re already using. North One was built to make managing money easy so you can focus on running your business. No more branches. No more lines. No more paperwork. Get complete access to your North One Account from your phone or computer, wherever your business takes you. Create Envelopes for taxes, payroll, rent, and anything else automatically.
-
10
nkn
Official Go implementation of NKN full node
NKN (New Kind of Network) is a blockchain-based network infrastructure that utilizes a novel consensus mechanism to facilitate decentralized data transmission. By incentivizing network participants with tokens, NKN aims to create a more open, efficient, and secure internet. Its architecture supports a wide range of applications, from messaging to content delivery. -
11
Prometheus
Open source monitoring system and time series database
Prometheus is a leading open source systems and service monitoring solution. It works by collecting metrics from configured targets at given intervals, evaluating rule expressions, and then displaying the results. It can also signal an alert if a condition is observed to be true. What sets Prometheus apart from other monitoring systems is its highly dimensional data model, powerful query language, autonomous single server nodes, among many other distinguishing features. It also offers... -
12
syslog-ng
Log management solution that improves the performance of SIEM
...Instead of deploying multiple agents on hosts, organizations can unify their log data collection and management. syslog-ng Store Box provides automated archiving, tamper-proof encrypted storage, granular access controls to protect log data. The largest appliance can store up to 10TB of raw logs. -
13
OpenNetMeter
A simple program to monitor your network/data usage
...It lives in the system tray, making it accessible and minimally intrusive for everyday use. Since it's open-source, users can inspect how data collection is done and trust that it’s not sending data elsewhere. While still evolving, it serves a practical niche for users who want more transparency than what built-in OS tools provide. -
14
Performance Co-Pilot (PCP)
Performance Co-Pilot
Performance Co-Pilot (PCP) provides a framework and services to support system-level performance monitoring and management. It presents a unifying abstraction for all of the performance data in a system, and many tools for interrogating, retrieving and processing that data. PCP is a feature-rich, mature, extensible, cross-platform toolkit supporting both live and retrospective analysis. The distributed PCP architecture makes it especially useful for those seeking centralized monitoring of... -
15
mtail
Extract internal monitoring data from application logs
Extract internal monitoring data from application logs for collection in a time-series database. mtail is a tool for extracting metrics from application logs to be exported into a timeseries database or timeseries calculator for alerting and dashboarding. It fills a monitoring niche by being the glue between applications that do not export their own internal state (other than via logs) and existing monitoring systems, such that system operators do not need to patch those applications to instrument them or writing custom extraction code for every such application. ... -
16
OpenTelemetry
OpenTelemetry Go API and SDK
OpenTelemetry-Go is the Go implementation of OpenTelemetry. It provides a set of APIs to directly measure the performance and behavior of your software and send this data to observability platforms. High-quality, ubiquitous, and portable telemetry to enable effective observability. OpenTelemetry is a collection of APIs, SDKs, and tools. Use it to instrument, generate, collect, and export telemetry data (metrics, logs, and traces) to help you analyze your software’s performance and behavior. -
17
AWS IoT FleetWise Edge
AWS IoT FleetWise Edge Agent
Easily collect, transform, and transfer vehicle data to the cloud in near-real-time. AWS IoT FleetWise makes it easy and cost-effective for automakers to collect, transform, and transfer vehicle data to the cloud in near-real-time and use it to build applications with analytics and machine learning that improve vehicle quality, safety, and autonomy. Train autonomous vehicles (AVs) and advanced driver assistance systems (ADAS) with camera data collected from a fleet of production vehicles.... -
18
QuestDB
An open source SQL database designed to process time series data
...It includes endpoints for PostgreSQL wire protocol, high-throughput schema-agnostic ingestion using InfluxDB Line Protocol, and a REST API for queries, bulk imports, and exports. QuestDB implements ANSI SQL with native extensions for time-oriented language features. These extensions make it simple to correlate data from multiple sources using relational and time series joins. QuestDB achieves high performance from a column-oriented storage model, massively-parallelized vector execution, SIMD instructions, and various low-latency techniques. The entire codebase was built from the ground up in Java and C++, with no dependencies, and is 100% free from garbage collection. ... -
19
File Converter
Simple tool which allows you to convert and compress files
File Converter is a minimalist open‑source tool (GPL‑3.0) that lets users convert and compress one or multiple files directly via the Windows Explorer context menu. It integrates with powerful back-end utilities—FFmpeg, ImageMagick, Ghostscript—to handle a broad range of media and document transformations. File Converter is a personal open source project started in 2014. I have put hundreds of hours adding, refining and tuning File Converter with the goal of making the conversion and... -
20
Sonobuoy
Sonobuoy is a diagnostic tool to understand Kubernetes cluster
Sonobuoy is a diagnostic tool that makes it easier to understand the state of a Kubernetes cluster by running a set of plugins (including Kubernetes conformance tests) in an accessible and non-destructive manner. It is a customizable, extendable, and cluster-agnostic way to generate clear, informative reports about your cluster. -
21
Grafana Agent
Vendor-neutral programmable observability pipelines
Grafana Agent is an OpenTelemetry Collector distribution with a configuration inspired by Terraform. It is designed to be flexible, performant, and compatible with multiple ecosystems such as Prometheus and OpenTelemetry. Grafana Agent is based on components. Components are wired together to form programmable observability pipelines for telemetry collection, processing, and delivery. -
22
Boehm-Demers-Weiser Garbage Collector
The Boehm-Demers-Weiser conservative C/C++ Garbage Collector
The Boehm-Demers-Weiser conservative garbage collector can be used as a garbage collecting replacement for C malloc or C++ new. It allows you to allocate memory basically as you normally would, without explicitly deallocating memory that is no longer useful. The collector automatically recycles memory when it determines that it can no longer be otherwise accessed. The collector is also used by a number of programming language implementations that either use C as intermediate code, want to... -
23
RyTuneX
RyTuneX is a cutting-edge optimizer built with the WinUI 3 framework
...It also offers system tweaks to improve responsiveness, reduce background overhead, and optimize system settings for gaming or heavy usage workflows. Privacy enhancements are part of its core offering: toggles for telemetry, data-collection features, background apps and more give users greater control over what their OS is tracking or doing under the hood. -
24
Breakpad
Mirror of Google Breakpad project
...Its flexible architecture enables developers to automate crash collection and reporting pipelines, making it a popular choice for large-scale software projects and products. -
25
Apache SkyWalking
Application Performance Monitoring System
Apache SkyWalking is an open source application performance monitoring system designed specifically for microservices, as well as cloud-native and container-based(Docker, Mesos, Kubernetes) architectures. It is capable of monitoring, tracing and diagnosing distributed systems in cloud native architectures. Apache Skywalking supports the collection of telemetry data from a number of different sources and in different formats, such as Java, .NET Core, PHP, LUA agent and more. It also offers a solid core features set, which includes root cause analysis, service topology map analysis, performance optimization, and many more.
