Search Results for "big data"
Sort By:
Showing 24 open source projects for "big data"
View related business solutions-
$300 Free Credits to Build on Google CloudStart your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
-
Stop Cyber Threats with VM-Series Next-Gen Firewall on AzureGain integrated visibility across all traffic in a single pass. Deploy Palo Alto Networks VM-Series to determine application identity and content while automating security policy updates via rich APIs.
-
1
Apache HBase
Get random, realtime read/write access to your Big Data
Use Apache HBase™ when you need random, realtime read/write access to your Big Data. This project's goal is the hosting of very large tables, billions of rows X millions of columns, atop clusters of commodity hardware. Apache HBase is an open-source, distributed, versioned, non-relational database modeled after Google's Bigtable. A Distributed Storage System for Structured Data by Chang et al. Just as Bigtable leverages the distributed data storage provided by the Google File System, Apache HBase provides Bigtable-like capabilities on top of Hadoop and HDFS. ... -
2
ODD Platform
First open-source data discovery and observability platform
Unlock the power of big data with OpenDataDiscovery Platform. Experience seamless end-to-end insights, powered by unprecedented observability and trust - from ingestion to production - while building your ideal tech stack! Democratize data and accelerate insights. Find data that fits your use case and discover hints left by your peers to leverage existing knowledge. -
3
Fluid
Fluid, elastic data abstraction and acceleration for BigData/AI apps
Fluid, elastic data abstraction and acceleration for BigData/AI applications in the cloud. Provide DataSet abstraction for underlying heterogeneous data sources with multidimensional management in a cloud environment. Enable dataset warmup and acceleration for data-intensive applications by using a distributed cache in Kubernetes with observability, portability, and scalability. Taking characteristics of application and data into consideration for cloud application/dataset scheduling to... -
4
Apache RocketMQ
Distributed messaging and streaming platform with low latency
...A variety of cross language clients, such as Java, C/C++, Python, Go. Pluggable transport protocols, such as TCP, SSL, AIO. Built-in message tracing capability, also support opentracing. Versatile big-data and streaming ecosytem integration. Message retroactivity by time or offset. Reliable FIFO and strict ordered messaging in the same queue. Efficient pull and push consumption model. Million-level message accumulation capacity in a single queue. Multiple messaging protocols like JMS and OpenMessaging. Flexible distributed scale-out deployment architecture. ... -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
BFG Repo-Cleaner
Remove large or troublesome blobs
The BFG is a simpler, faster alternative to git-filter-branch for cleansing bad data out of your Git repository history. You can use it for removing crazy big files, and for removing passwords, credentials and other private data. The git-filter-branch command is enormously powerful and can do things that the BFG can't, but the BFG is much better for the tasks above, because is faster and simpler. The BFG isn't particularily clever, but is focused on making the above tasks easy. ... -
6
Grafana Alloy
OpenTelemetry Collector distribution with programmable pipelines
Grafana Alloy is an open source OpenTelemetry Collector distribution with built-in Prometheus pipelines and support for metrics, logs, traces, and profiles. Grafana Alloy is Grafana Labs’ distribution of the OpenTelemetry Collector. It is an OTLP-compatible collector with built-in Prometheus optimizations that also support signals across metrics, logs, traces, and profiles. Alloy was started at Grafana Labs and announced at GrafanaCON in 2024. The mission of the project is to create the best... -
7
LakeSoul
An end-to-end, realtime and cloud native Lakehouse framework
LakeSoul is a high-performance, unified table storage framework for big data lakes, supporting both streaming and batch data in a single format. Built on top of Apache Spark and leveraging Apache Arrow and Parquet, LakeSoul provides ACID transactions, schema evolution, and time travel. It is designed for large-scale data lake architectures that require consistency, efficiency, and easy integration with modern data stacks. -
8
manticoresearch
Easy to use open source fast database for search
...Modern MPP architecture and smart query parallelization capabilities allow to fully utilize all your CPU cores to lower response time as much as possible, when needed. Powerful and fast full-text searching which works fine for small and big datasets. Columnar storage support via the Manticore Columnar Library for bigger datasets (much bigger than can fit in RAM). SQL-first: Manticore's native syntax is SQL. It speaks SQL over HTTP and uses the MySQL protocol (you can use your preferred MySQL client). JSON over HTTP: to provide a more programmatic way to manage your data and schemas, Manticore provides a HTTP JSON protocol. ... -
9
StartOS
Linux server OS optimized for self-hosting
StartOS is a sovereign, self-hosted operating system built by Start9 Labs to empower individuals with digital independence. Designed to run on personal servers, it provides a privacy-first interface for installing, managing, and running decentralized applications without needing technical expertise. StartOS includes services like Bitcoin nodes, messaging platforms, file hosting, and password managers, all running locally and without third-party control. With a user-friendly UI and strong... -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
10
MobX
A Simple, scalable state management
MobX is a battle tested library that makes state management simple and scalable by transparently applying functional reactive programming (TFRP). Write minimalistic, boilerplate free code that captures your intent. Trying to update a record field? Use the good old JavaScript assignment. Updating data in an asynchronous process? No special tools are required, the reactivity system will detect all your changes and propagate them out to where they are being used. All changes to and uses of... -
11
IPFS
IPFS implementation in Go
A peer-to-peer hypermedia protocol designed to make the web faster, safer, and more open. HTTP downloads files from one computer at a time instead of getting pieces from multiple computers simultaneously. Peer-to-peer IPFS saves big on bandwidth, up to 60% for video, making it possible to efficiently distribute high volumes of data without duplication. The average lifespan of a web page is 100 days before it's gone forever. It's not good enough for the primary medium of our era to be this fragile. IPFS keeps every version of your files and makes it simple to set up resilient networks for mirroring data. ... -
12
Foundatio
Pluggable foundation blocks for building distributed apps
Pluggable foundation blocks for building loosely coupled distributed apps. Includes implementations in Redis, Azure, AWS, RabbitMQ and in memory (for development). When building several big cloud applications we found a lack of great solutions (that's not to say there aren't solutions out there) for many key pieces to building scalable distributed applications while keeping the development experience simple. Wanted to build against abstract interfaces so that we could easily change... -
13
json-scada
A portable SCADA/IoT platform centered on the MongoDB database server.
...MongoDB as the real-time core database, persistence layer, config store, SOE historian. Portability and interoperability over Linux, Windows, x86/64, ARM. Horizontal scalability, from a single computer to big clusters (MongoDB-sharding), Bare Metal, Docker containers, VM, cloud, or hybrid deployments. Unlimited tags, servers, and users. HTML5 Web interface. UTF-8/I18N. Protocols: IEC61850 Client, IEC60870-5-101/104 Client and Server, DNP3 Client, OPC-UA Client/Server, MQTT/Sparkplug-B, Telegraf (various data sources for monitoring like Modbus, SNMP, etc.) ... -
14
Curve
Curve is a sandbox project hosted by the CNCF Foundation
A cloud-native distributed storage system. A sandbox project hosted by the CNCF Foundation. Curve is a modern storage system developed by netease, currently supporting file storage(CurveFS) and block storage(CurveBS). Now it's hosted at CNCF as a sandbox project. The performance, mixed, capacity cloud disk or persistent volume of virtual machine/container, and remote disks of physical machines. High-performance separation of storage and computation architecture: high-performance and low... -
15
Koha + DSpace 10.0 Live ISO (2026.06)
Koha 26.05.00 + DSpace 10.0 Live ISO Installer based on Ubuntu 22.04.5
Koha + DSpace Live is a Live Bootable and Installer ISO based on Ubuntu 22.04.5 (Koha Version: 26.05.00 and DSpace Version: 10.0) This ISO boots only in Legacy BIOS mode and not in Secure Boot Mode After booting from the Live DVD/USB, use the following login credentials: Login: library (Displayed as 'Open Digital Library') Password: library This Live ISO contains additional Desktop Environments (aka D.E) providing different User Interfaces. If you wish to choose a Lightweight... -
16
DataSophon
The next generation of cloud-native big data management expert
Aiming at quickly deploying, managing, monitoring and automating the operation and maintenance of Big Data service components and nodes, helping you quickly build stable, efficient Big Data cluster services. The Three-Body Problem, a Hugo Award-winning work of the world's highest science fiction literature, is known for its stunning "hard science fiction" style, and its author Liu Cixin is credited with "single-handedly raising Chinese science fiction to a world-class level". ... -
17
GnuCopy is an Open-Source tool to copy and archive all your important data. It supports all important archive typs like Zip and Tar to guaranty an easy and secure exchange between all types of operating systems. Additionally, you can create profiles to blacklist or whitelist specific file types or folders to seperate your big data stores for backups.
-
18
Annoy
Approximate Nearest Neighbors in C++/Python optimized for memory usage
Annoy (Approximate Nearest Neighbors Oh Yeah) is a C++ library with Python bindings to search for points in space that are close to a given query point. It also creates large read-only file-based data structures that are mmapped into memory so that many processes may share the same data. There are some other libraries to do nearest neighbor search. Annoy is almost as fast as the fastest libraries, (see below), but there is actually another feature that really sets Annoy apart: it has the... -
19
MC 6800 Emulator for Teensy 4
6800 Emulator runs on Teensy 4 processor
6800/ET-3400 Emulator, designed for the Teensy 4.0/4.1 processors. Build this with the Arduino IDE 2.0. Full Source and readme included in archive. Programs written in 6800 assembly can be run in the emulator. It supports two modes of operation, an internal monitor program and a mode that emulates the Heathkit ET3400 (and ET-6800) microprocessor trainers. Motorola released the MC6800 chip in 1974. It was a source of inspiration for the designers of the more popular (and cheaper) 6502 chip... -
20
wzd
Powerful storage server, designed for big data storage systems
wZD is a server written in Go language that uses a modified version of the BoltDB database as a backend for saving and distributing any number of small and large files, NoSQL keys/values, in a compact form inside micro Bolt databases (archives), with distribution of files and values in BoltDB databases depending on the number of directories or subdirectories and the general structure of the directories. Using wZD can permanently solve the problem of a large number of files on any POSIX... -
21
Visual Disk Diet
Helps you visualize the space used on your drives with colorful chart.
...Feel free to use, modify or suggest ideas! It's pretty much a copy of Disk Space Fan but Open Source so it can be adapted to extended purposes (as browsing file system...) It's very fast on data drives (your data partition, usb drive...) It's NOT optimized for very large and complex drives such as big C: drive for now so it can take up to 10 minutes to scan it (and there is no feedback during this scan, so it can be frustrating). -
22
AIS - Associative Indexing Service, an application for storing bookmarks, memos, indexing of big (lifetime) archives for fast future access to the data by (personalized) keywords. In other words - it is an extension of human associative memory :)
-
23
Cube Platform is a decentralized grid computing system that uses P2P Pastry protocol for communication between nodes. It's a big data storage written in Java.
-
24
a 3d replacement for mosix's boring text based mon program. Interaces with gomd (www.nonngu.org/gomd) so you dont have to run this big openGL app on your cluster, just a handy server.
- Previous
- You're on page 1
- Next
