Search Results for "disk partitioning and data management"
Sort By:
Showing 116 open source projects for "disk partitioning and data management"
View related business solutions-
Go From AI Idea to AI App FastAccess Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
-
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
1
pg_partman
Partition management extension for PostgreSQL
pg_partman is a PostgreSQL extension for managing time-based and serial-based table partitioning. It automates the creation, maintenance, and retention of partitions, making it easier to manage large datasets with better performance and query efficiency. Designed for both native and trigger-based partitioning, pg_partman is ideal for applications handling time-series or log data. -
2
Redis
An in-memory database that persists on disk
Redis is an open-source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Redis provides data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Redis has built-in replication, Lua scripting, LRU eviction, transactions, and different levels of on-disk persistence, and provides high availability via Redis Sentinel and automatic partitioning with Redis Cluster. ... -
3
hekate
A GUI based Nintendo Switch Bootloader
...It provides advanced storage management capabilities such as backing up and restoring eMMC data, managing emulated NAND environments, and partitioning SD cards for multiple operating systems. hekate also includes utilities for hardware diagnostics, benchmarking, and system inspection, offering detailed insights into device components like memory, battery, and storage. -
4
Space Radar Electron
Disk And Memory Space Visualization App built with Electron & d3.js
Space Radar Electron is an application that offers an interactive and comprehensive visualization of disk space and memory usage of your computer. Built with Electron & d3.js, it currently offers visualizations in the form of Sunburst, Treemap and Flamegraph charts. As it scans the contents of your disk, it produces a preview visualization so you can already see what's been scanned. It allows for drilldown of directories, breadcrumbs and navigation. Space Radar works fast, and is... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
Zenith
Sort of like top or htop but with zoom-able charts, CPU, GPU
In terminal graphical metrics for your *nix system written in Rust. The make file provides for building fully static versions on Linux against the musl C library. It requires musl-gcc to be installed on the system. Install "musl-tools" package on debian/ubuntu derivatives, "musl-gcc" on fedora and equivalent on other distributions from their standard repos. If one needs to build with NVIDIA support in a virtual environment, then it requires some more setup since typically the VM software is... -
6
JDF.jl
Julia DataFrames serialization format
JDF is a DataFrames serialization format with the following goals, fast save and load times, compressed storage on disk, enabled disk-based data manipulation (not yet achieved), and support for machine learning workloads, e.g. mini-batch, sampling (not yet achieved). JDF stores a DataFrame in a folder with each column stored as a separate file. There is also a metadata.jls file that stores metadata about the original DataFrame. Collectively, the column files, the metadata file, and the... -
7
ReductStore
The fastest time series object store for Edge AI
...Outperform other databases with a customized solution for time-series object data. Capture and access blob data as time series, tailored for edge computing, computer vision, and IoT. ReductStore handles blob data without size limits; your disk capacity is the only boundary. Ensure optimal storage management with FIFO quotas, preventing disk space shortages in real-time. Manage your time-series blob data with ease: annotate, filter, and save AI labels or meta-data. ... -
8
PlantUML
Generate diagrams from textual description
Generate UML diagram from textual description. PlantUML is not affected by the log4j vulnerability. The easiest way to test PlantUML is in an online solution that has PlantUML embedded, such as our online server. After testing, you may want to install PlantUML locally. Run (or have your software call) PlantUML, using sequenceDiagram.txt as input. The output is an image, which either appears in the other software, or is written to an image file on disk. Diagrams are defined using a simple and... -
9
IoTDB
Apache IoTDB
Apache IoTDB (Database for Internet of Things) is an IoT native database with high performance for data management and analysis, deployable on the edge and the cloud. Due to its light-weight architecture, high performance and rich feature set together with its deep integration with Apache Hadoop, Spark and Flink, Apache IoTDB can meet the requirements of massive data storage, high-speed data ingestion and complex data analysis in the IoT industrial fields. In the scene of factories, there... -
AI-generated apps that pass security reviewRetool lets you generate dashboards, admin panels, and workflows directly on your data. Type something like “Build me a revenue dashboard on my Stripe data” and get a working app with security, permissions, and compliance built in from day one. Whether on our cloud or self-hosted, create the internal software your team needs without compromising enterprise standards or control.
-
10
GenStage
Producer and consumer actors with back-pressure for Elixir
GenStage is a specification and set of behaviours for building demand-driven data pipelines on the BEAM. It formalizes the roles of producers, consumers, and producer-consumers, using back-pressure so that fast producers don’t overwhelm downstream stages. Developers implement callbacks like handle_demand and handle_events to control how items are emitted, transformed, and consumed across asynchronous boundaries. Because stages are OTP processes, you gain fault tolerance, supervised restarts,... -
11
beszel
Lightweight server monitoring hub with historical data, docker stats
A lightweight server resource monitoring hub with historical data, docker stats, and alerts. -
12
fluentbit
Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX
...It is the preferred choice for cloud and containerized environments. A robust, lightweight, and portable architecture for high throughput with low CPU and memory usage from any data source to any destination. Proven across distributed cloud and container environments. Highly available with I/O handlers to store data for disaster recovery. Granular management of data parsing and routing. Filtering and enrichment to optimize security and minimize cost. The lightweight, asynchronous design optimizes resource usage: CPU, memory, disk I/O, network. ... -
13
AWS SDK for pandas
Easy integration with Athena, Glue, Redshift, Timestream, Neptune
aws-sdk-pandas (formerly AWS Data Wrangler) bridges pandas with the AWS analytics stack so DataFrames flow seamlessly to and from cloud services. With a few lines of code, you can read from and write to Amazon S3 in Parquet/CSV/JSON/ORC, register tables in the AWS Glue Data Catalog, and query with Amazon Athena directly into pandas. The library abstracts efficient patterns like partitioning, compression, and vectorized I/O so you get performant data lake operations without hand-rolling... -
14
FileTrees.jl
Parallel file processing made easy
Easy everyday parallelism with a file tree abstraction. Read a directory structure as a Julia data structure, (lazy-)load the files, apply map and reduce operations on the data while not exceeding available memory if possible. Make up a file tree in memory, create some data to go with each file (in parallel), write the tree to disk (in parallel). FileTrees is a set of tools to lazy-load, process and save file trees. Built-in parallelism allows you to max out all threads and processes that... -
15
GridDB
GridDB is a next-generation open source database
...Multi-model architecture capable of supporting various data stores with time-series data-oriented and pluggable data stores for efficient real-time processing and management of huge amounts of time-series data at high frequency. Various architectural innovations, such as in-memory orientation with "memory as the main unit and disk as the secondary unit" and event-driven design with minimal overhead, have been incorporated to achieve processing capabilities that can handle petabyte-scale applications. -
16
ClickHouse
A fast open-source OLAP database management system
ClickHouse® is a fast, open-source column-oriented database management system that can generate analytical data reports through SQL queries in real time. According to several independent benchmarks, it far exceeds other comparable column-oriented database management systems, working even up to 1000 times faster. It is able to process hundreds of millions to more than a billion rows and tens of gigabytes of data per single server per second. Apart from its blazing speed, ClickHouse is highly... -
17
Benthos
Fancy stream processing made operationally mundane
Benthos is a high performance and resilient stream processor, able to connect various sources and sinks in a range of brokering patterns and perform hydration, enrichments, transformations and filters on payloads. It comes with a powerful mapping language, is easy to deploy and monitor, and ready to drop into your pipeline either as a static binary, docker image, or serverless function, making it cloud native as heck. Delivery guarantees can be a dodgy subject. Benthos processes and... -
18
Spring Batch
Spring Batch is a framework for writing batch applications using Java
A lightweight, comprehensive batch framework designed to enable the development of robust batch applications vital for the daily operations of enterprise systems. Spring Batch provides reusable functions that are essential in processing large volumes of records, including logging/tracing, transaction management, job processing statistics, job restart, skip, and resource management. It also provides more advanced technical services and features that will enable extremely high-volume and high... -
19
Checkmate
Checkmate is an open-source, self-hosted tool
Checkmate is an open-source, self-hosted infrastructure monitoring platform that provides real-time visibility into server health, uptime, response times, and incident activity through a modern web interface. The application continuously checks whether websites and services are accessible and performing optimally, generating alerts and reports when availability or performance degrades. It supports detailed infrastructure monitoring through an optional agent called Capture, which collects... -
20
disk-sentinel
Lightweight Linux daemon that prevents unnecessary HDD spin-ups using
...Unlike traditional power management tools, Disk Sentinel focuses on a NO-WAKEUP philosophy: it avoids polling or monitoring methods that accidentally spin disks up, ensuring they stay truly idle until real access is required. -
21
MemProcFS Analyzer
Automated Forensic Analysis of Windows Memory Dumps for DFIR
MemProcFS-Analyzer is a forensic analysis toolset that builds on the MemProcFS virtual filesystem to make volatile memory artefacts easier to browse and interpret. By exposing process memory, kernel objects, and derived artifacts as regular files, the framework lets analysts use familiar filesystem operations and standard tools (editors, grep, diff) to explore memory snapshots. The Analyzer layer adds higher-level parsing and extraction routines—for example, carving strings, locating... -
22
Hard Disk Sentinel Professional
Pro hard drive monitoring, disk health & temperature monitoring tool
...With the Professional edition, users gain access to more features like scheduled backup functionality, remote monitoring, and advanced disk management. It’s ideal for both home users and IT professionals seeking to maintain optimal disk health and prevent hardware failures. -
23
Geziyor
Blazing fast Go framework for web crawling and data scraping tasks
Geziyor is a high-performance web crawling and web scraping framework built for the Go programming language. It is designed to help developers crawl websites and extract structured information from web pages efficiently. It focuses on speed and scalability, allowing large numbers of requests to be processed concurrently. Geziyor supports use cases such as data mining, monitoring web content, and automated testing workflows. It provides a flexible architecture where developers define parsing... -
24
nb-clean
Clean Jupyter notebooks of outputs, metadata, and empty cells
nb-clean cleans Jupyter notebooks of cell execution counts, metadata, outputs, and (optionally) empty cells, preparing them for committing to version control. It provides both a Git filter and pre-commit hook to automatically clean notebooks before they're staged, and can also be used with other version control systems, as a command line tool, and as a Python library. It can determine if a notebook is clean or not, which can be used as a check in your continuous integration pipelines.... -
25
Vortex
The LLVM of columnar file formats
Vortex is a high-performance toolkit designed for working with compressed Apache Arrow arrays, providing functionality for in-memory, on-disk, and over-the-wire data handling. It aims to be an advanced successor to Apache Parquet, offering dramatically faster random access reads and scans, while maintaining similar compression ratios. Vortex's modular design allows for extensibility, enabling developers to implement custom encodings for efficient data management, particularly for large-scale columnar datasets.
