Search Results for "cloud storage"
Sort By:
Showing 80 open source projects for "cloud storage"
View related business solutions-
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
cloud_enum
Multi-cloud OSINT tool for discovering public cloud resources
cloud_enum is an open source reconnaissance and OSINT tool designed to discover publicly accessible cloud resources across major cloud providers. It focuses on enumerating assets in Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform using keyword-based discovery techniques. It works by taking user-provided keywords and generating variations through mutation wordlists, then testing these combinations against common cloud service naming patterns. cloud_enum performs both HTTP probing and DNS lookups to identify resources such as storage buckets, cloud applications, and databases that may be exposed or accessible. cloud_enum uses concurrent processing to speed up scanning, enabling efficient enumeration of large numbers of possible resource names. ... -
2
Cloud Custodian
Rules engine for cloud security, cost optimization, and governance
Cloud Custodian enables users to be well managed in the cloud. The simple YAML DSL allows you to easily define rules to enable a well-managed cloud infrastructure, that's both secure and cost-optimized. It consolidates many of the ad-hoc scripts organizations have into a lightweight and flexible tool, with unified metrics and reporting. Custodian supports managing AWS, Azure, and GCP public cloud environments. Besides just providing reports of issues, Custodian can actively enforce the... -
3
PGHoard
PostgreSQL® backup and restore service
pghoard is a PostgreSQL backup and restore tool that provides encrypted, compressed, and cloud-optimized backups. Developed by Aiven, it supports streaming WAL archiving and full base backups to various cloud storage backends. pghoard is designed for reliability and fast disaster recovery in cloud-native PostgreSQL deployments. -
4
Google Cloud Platform Python Samples
Code samples used on cloud.google
Google Cloud Platform Python Samples repository is a large, curated collection of Python code examples that demonstrate how to use a wide range of Google Cloud services in real-world scenarios. It serves as a practical companion to official documentation, providing runnable snippets that illustrate how to authenticate, configure environments, and interact with APIs across products such as storage, AI services, and data processing tools. -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
5
Lithops
A multi-cloud framework for big data analytics
...It also supports hybrid cloud setups, object storage access, and simple integration with Jupyter notebooks. -
6
Firebase Admin Python SDK
Firebase Admin Python SDK
Firebase provides the tools and infrastructure you need to develop apps, grow your user base, and earn money. The Firebase Admin Python SDK enables access to Firebase services from privileged environments (such as servers or cloud) in Python. Currently this SDK provides Firebase custom authentication support. Create your own simplified admin console to do things like look up user data or change a user's email address for authentication. Access Google Cloud resources like Cloud Storage buckets and Cloud Firestore databases associated with your Firebase projects. ... -
7
Databend
Cloud-native open source data warehouse for analytics and AI queries
Databend is an open source cloud-native data warehouse designed for large-scale analytics and modern data workloads. Built in Rust, the system focuses on high performance, scalability, and efficient data processing for analytical queries. It is designed with a separation of compute and storage, allowing compute nodes to scale independently while storing data in object storage systems. -
8
DVC
Data Version Control | Git for Data & Models
...It is designed to handle large files, data sets, machine learning models, and metrics as well as code. Version control machine learning models, data sets and intermediate files. DVC connects them with code and uses Amazon S3, Microsoft Azure Blob Storage, Google Drive, Google Cloud Storage, Aliyun OSS, SSH/SFTP, HDFS, HTTP, network-attached storage, or disc to store file contents. Version control machine learning models, data sets, and intermediate files. DVC connects them with code and uses Amazon S3, Microsoft Azure Blob Storage, Google Drive, Google Cloud Storage, Aliyun OSS, SSH/SFTP, HDFS, HTTP, network-attached storage, or disc to store file contents. ... -
9
smart_open
Utils for streaming large files (S3, HDFS, gzip, bz2...)
smart_open is a Python library for streaming very large files from local or remote storage through a familiar file-like interface. It acts as an extended replacement for Python's built-in open function. Supported backends include Amazon S3, Google Cloud Storage, Azure Blob Storage, HDFS, WebHDFS, HTTP, HTTPS, SFTP, and local filesystems. Data can be read or written without first downloading an entire remote object to disk. -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
diskover-community
Open source file indexing & storage analytics powered by Elasticsearch
...By indexing file metadata from sources such as local file systems, network shares like NFS and SMB, and cloud storage, the tool provides a centralized way to analyze heterogeneous storage environments. Diskover also helps identify outdated or unused files, duplicate data, and inefficient storage usage that can waste resources or increase operational costs. A Python-based indexing engine performs the scanning and indexing tasks. -
11
Deep Lake
Data Lake for Deep Learning. Build, manage, and query datasets
...Our open-source dataset format is optimized for rapid streaming and querying of data while training models at scale, and it includes a simple API for creating, storing, and collaborating on AI datasets of any size. It can be deployed locally or in the cloud, and it enables you to store all of your data in one place, ranging from simple annotations to large videos. Deep Lake is used by Google, Waymo, Red Cross, Omdena, Yale, & Oxford. Use one API to upload, download, and stream datasets to/from AWS S3/S3-compatible storage, GCP, Activeloop cloud, or local storage. Store images, audios and videos in their native compression. ... -
12
MicroK8s
Single-package Kubernetes for developers, IoT and edge
Low-ops, minimal production Kubernetes, for devs, cloud, clusters, workstations, Edge and IoT. MicroK8s automatically chooses the best nodes for the Kubernetes datastore. When you lose a cluster database node, another node is promoted. No admin needed for your bulletproof edge. MicroK8s is small, with sensible defaults that ‘just work’. A quick install, easy upgrades and great security make it perfect for micro clouds and edge computing. As the publishers of MicroK8s, we deliver the world’s... -
13
Unstract
No-code LLM Platform to launch APIs and ETL Pipelines
...Unstract supports deploying structured extraction as REST API endpoints or embedding it into data engineering ETL pipelines, which allows it to plug directly into data warehouses, cloud storage, or downstream analytics systems. Its platform works with a broad variety of file types — from PDFs and spreadsheets to images — and includes integrations with databases, cloud storage providers, and vector databases. -
14
Cookiecutter Django
Framework for jumpstarting production-ready Django projects quickly
...Provides an optional basic ASGI setup for Websockets and an optional custom static build using Gulp and livereload. Send emails via Anymail (using Mailgun by default or Amazon SES if AWS is selected cloud provider, but switchable). Media storage using Amazon S3 or Google Cloud Storage. Docker support using docker-compose for development and production (using Traefik with LetsEncrypt support). Procfile for deploying to Heroku. Provides instructions for deploying to PythonAnywhere. You can run tests with unittest or pytest. -
15
django-dbbackup
Management commands to help backup and restore your project database
django-dbbackup is a Django management command extension for backing up and restoring databases and media files. It supports multiple storage backends including local file systems, Amazon S3, and Dropbox. Ideal for automated backup strategies, it simplifies the integration of backup logic into Django projects. -
16
CoPaw
Your Personal AI Assistant; easy to install, deploy on local or coud
CoPaw is a personal AI assistant designed to run on your own machine or in the cloud, giving you full control over memory, models, and data. Built by the AgentScope team, it connects to multiple chat platforms—including DingTalk, Feishu, QQ, Discord, iMessage, and more—through a single unified assistant. CoPaw supports both cloud-based LLM providers and fully local models such as llama.cpp, MLX, and Ollama, allowing you to operate without API keys if preferred. It includes a browser-based... -
17
Django-CRM
Open Source CRM based on Django
Django CRM is opensource CRM developed on django framework. It has all the basic features of CRM to start with. We welcome code contributions and feature requests via github. Create and activate a virtual environment. Install the project's dependency after activating env. -
18
Memvid
Video-based AI memory library. Store millions of text chunks in MP4
Memvid encodes text chunks as QR codes within MP4 frames to build a portable “video memory” for AI systems. This innovative approach uses standard video containers and offers millisecond-level semantic search across large corpora with dramatically less storage than vector DBs. It's self-contained—no DB needed—and supports features like PDF indexing, chat integration, and cloud dashboards. -
19
pgai
A suite of tools to develop RAG, semantic search, and other AI apps
pgai is a suite of PostgreSQL extensions developed by Timescale to empower developers in building AI applications directly within their databases. It integrates tools for vector storage, advanced indexing, and AI model interactions, facilitating the development of applications like semantic search and Retrieval-Augmented Generation (RAG) without leaving the SQL environment. -
20
MemMachine
Universal memory layer for AI Agents
MemMachine is a universal memory layer designed for AI agents that provides persistent, rich memory storage and retrieval capabilities so autonomous agent systems can recall context, personal preferences, and long-term interaction history across sessions, models, and use cases. Unlike ephemeral LLM prompt state, MemMachine supports distinct memory types—short-term conversational context, long-term persistent knowledge, and profile memory for personalized facts—persisted in optimized stores... -
21
Meli-Action
Download files from direct links, YouTube, Telegram, Google Play
Meli-Action is a GitHub Actions-based downloading and web archiving project designed for heavily restricted internet environments. It uses GitHub-hosted workflow runners to fetch content from direct links, YouTube, Telegram, Google Play, SoundCloud, and web pages. Downloaded files are committed into the user’s repository so they can be retrieved later through normal GitHub access. The project includes workflows for different content sources, plus a Python script that can render and save web... -
22
MemPalace
The highest-scoring AI memory system ever benchmarked
MemPalace is an open-source AI memory system designed to solve one of the most persistent limitations of large language models: the loss of context between sessions. Instead of relying on summarization or selective extraction like most memory tools, it takes a radically different approach by storing conversations in their entirety and making them retrievable through structured organization and semantic search. The system is inspired by the classical “memory palace” mnemonic technique,... -
23
Factorio
Factorio headless server in a Docker container
Factorio is an open-source project that provides a Docker container for running a headless Factorio game server. Factorio is a factory-building simulation game in which players automate production lines, research technologies, and manage complex industrial systems, and the repository focuses specifically on hosting the game server in a containerized environment. By packaging the server into a Docker image, the project simplifies the process of deploying and maintaining multiplayer servers... -
24
MineContext
MineContext is your proactive context-aware AI partner
...Unlike traditional chat-based assistants, MineContext operates in the background and delivers proactive outputs such as daily summaries, task suggestions, and contextual reminders without requiring explicit prompts. It is built around a context engineering framework that manages the full lifecycle of data, including capture, processing, storage, retrieval, and consumption. The platform emphasizes privacy through a local-first architecture, allowing users to keep their data stored and processed on their own device rather than relying on external cloud services. -
25
QA-Pilot
QA-Pilot is an interactive chat project that leverages LLMs
...It enables users to clone repositories locally and then interact with them through a conversational interface, allowing for rapid exploration of codebases without manual searching. The system supports both local and cloud-based LLM providers, making it flexible for different environments and privacy requirements. It includes features such as chat session storage, search functionality for retrieving previous interactions, and multi-session management for working on multiple repositories simultaneously. QA-Pilot also integrates code visualization tools such as code graphs, helping users better understand file structures and relationships within projects. ...
