Search Results for "s3"
Sort By:
Showing 290 open source projects for "s3"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
Build Securely on Azure with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
1
S3 GUI
S3 GUI for Desktop, Mobile and Web
A Desktop, Mobile and Web client for S3 built with Flutter. -
2
s3-client
Sample python script to work with Amazon S3
Example Python script to work with S3. -
3
AsyncAws S3 Client
S3 client
An API client for S3. AsyncAws has implemented the most popular API clients. If you want to upload a 1 Gb file, you really don't want to put that file in memory before uploading. You want to do it a smarter way. AsyncAws allows you to upload files using a string, resource, closure, or an iterable. -
4
STU
TUI application for AWS S3 written in Rust using ratatui
TUI application for AWS S3 written in Rust using ratatui. -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
5
MinIO
High performance object storage server compatible with Amazon S3 APIs
MinIO is a high performance object storage server that is API compatible with Amazon S3 cloud storage service. MinIO makes it easy to build high performance, cloud native data infrastructure for machine learning, analytics and application data workloads. It is incredibly fast, enabling object storage to operate as the primary storage tier for a diverse set of workloads. It is also built to be cloud native and enterprise ready. -
6
MinIO Client Quickstart Guide
Fast tool to manage MinIO clusters
MinIO Client (mc) provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff, find etc. It supports filesystems and Amazon S3-compatible cloud storage service (AWS Signature v2 and v4). MinIO is distributed under GNU AGPLv3. See the source tree for complete information on all components of the work and their licenses. MinIO is a high-performance object storage solution that provides an Amazon Web Services S3-compatible API and supports all core S3 features. ... -
7
AWS Data Wrangler
Pandas on AWS, easy integration with Athena, Glue, Redshift, etc.
...Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON, and EXCEL). Built on top of other open-source projects like Pandas, Apache Arrow and Boto3, it offers abstracted functions to execute usual ETL tasks like load/unload data from Data Lakes, Data Warehouses, and Databases. Convert the column name to be compatible with Amazon Athena and the AWS Glue Catalog. Run a query against AWS CloudWatchLogs Insights and convert the results to Pandas DataFrame. ... -
8
smart_open
Utils for streaming large files (S3, HDFS, gzip, bz2...)
smart_open is a Python library for streaming very large files from local or remote storage through a familiar file-like interface. It acts as an extended replacement for Python's built-in open function. Supported backends include Amazon S3, Google Cloud Storage, Azure Blob Storage, HDFS, WebHDFS, HTTP, HTTPS, SFTP, and local filesystems. Data can be read or written without first downloading an entire remote object to disk. Transparent compression and decompression are supported for multiple common formats. Standard operations such as context managers, iteration, and supported seek behavior remain available. ... -
9
AWS SDK for PHP
Repository of the AWS SDK for PHP
The AWS SDK for PHP makes it easy for developers to access Amazon Web Services in their PHP code, and build robust applications and software using services like Amazon S3, Amazon DynamoDB, Amazon Glacier, etc. You can get started in minutes by installing the SDK through Composer or by downloading a single zip or phar file from our latest release. Before you begin, you need to sign up for an AWS account and retrieve your AWS credentials. To run the SDK, your system will need to meet the minimum requirements, including having PHP >= 5.5. ... -
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
10
S3Mock
A simple mock implementation of the AWS S3 API startable as Docker
S3Mock is a lightweight server that implements parts of the Amazon S3 API. It has been created to support local integration testing by reducing infrastructure dependencies. The S3Mock server can be started as a standalone Docker container, using Test containers, JUnit4, JUnit5 and TestNG support, or programmatically. -
11
CloudServer
Zenko CloudServer open-source Node.js implementation of S3 protocol
Zenko CloudServer, an open-source Node.js implementation of the Amazon S3 protocol on the front-end and backend storage capabilities to multiple clouds, including Azure and Google. -
12
MinIO Go Client SDK
MinIO Go client SDK for S3 compatible object storage
The MinIO Go Client SDK provides simple APIs to access any Amazon S3-compatible object storage. This quickstart guide will show you how to install the MinIO client SDK, connect to MinIO, and provide a walkthrough for a simple file uploader. For a complete list of APIs and examples, please take a look at the Go Client API Reference. -
13
S3Proxy
S3Proxy implements the S3 API and proxies requests
S3Proxy implements the S3 API and proxies requests, enabling several use cases. Translation from S3 to Backblaze B2, EMC Atmos, Google Cloud, Microsoft Azure, and OpenStack Swift. Testing without Amazon by using the local filesystem. Extension via middleware. Embedding into Java applications. Docker Hub hosts a Docker image and has instructions on how to run it. -
14
remotefs
A client library to work with all file transfer protocols
...This is achieved through a trait called RemoteFs which exposes methods to operate on the remote file system. Currently, the library exposes a client for Sftp, Scp, Ftp and Aws-s3 as external libraries. You might be wondering, "why remotefs?" The first and foremost reason is to provide a generic interface over multiple protocols. For example, in termscp it allows the support of multiple protocols without any protocol-specific code. The second reason is that often, users just want a simple way to operate on a remote file system, however, they don't have the time to spend researching the ins and outs of each protocol. -
15
s3fs
FUSE-based file system backed by Amazon S3
s3fs allows Linux, macOS, and FreeBSD to mount an S3 bucket via FUSE. s3fs preserves the native object format for files, allowing use of other tools like AWS CLI. Large subset of POSIX including reading/writing files, directories, symlinks, mode, uid/gid, and extended attributes. Compatible with Amazon S3, and other S3-based object stores. Allows random writes and appends, large files via multi-part upload, and renames via server-side copy. -
16
SFTPGo
Fully featured and highly configurable SFTP server with optional HTTP
Fully featured and highly configurable SFTP server with optional HTTP/S, FTP/S and WebDAV support. Several storage backends are supported: local filesystem, encrypted local filesystem, S3 (compatible) Object Storage, Google Cloud Storage, Azure Blob Storage, SFTP. SFTPGo is an Open Source project and you can of course use it for free but please don't ask for free support as well. Support for serving local filesystem, encrypted local filesystem, S3 Compatible Object Storage, Google Cloud Storage, Azure Blob Storage or other SFTP accounts over SFTP/SCP/FTP/WebDAV. ... -
17
S5CMD
Parallel S3 and local filesystem execution tool
s5cmd is a very fast S3 and local filesystem execution tool. It comes with support for a multitude of operations including tab completion and wildcard support for files, which can be very handy for your object storage workflow while working with large number of files. There are already other utilities to work with S3 and similar object storage services, thus it is natural to wonder what s5cmd has to offer that others don't. s5cmd supports wide range of object management tasks both for cloud storage services and local filesystems. ... -
18
Litestream
Streaming replication for SQLite
Litestream is a standalone disaster recovery tool for SQLite databases. It runs as a background process and continuously replicates SQLite changes to another file or remote storage such as S3. The project is designed to make SQLite more practical for production apps that need simple recovery without operating a full database server. It communicates through the SQLite API, which helps reduce the risk of corrupting the database. Litestream is written mainly in Go and includes CLI tooling, Docker support, tests, documentation, and storage-related packages. ... -
19
SageMaker TensorFlow Training Toolkit
Toolkit for running TensorFlow training scripts on SageMaker
...After creating a SageMaker Model, you can use it to create SageMaker Batch Transform Jobs for offline inference, or create SageMaker Endpoints for real-time inference. A SageMaker Model contains references to a model.tar.gz file in S3 containing serialized model data, and a Docker image used to serve predictions with that model. A Batch Transform job runs an offline-inference job using your TensorFlow Serving model. Input data in S3 is converted to HTTP requests, and responses are saved to an output bucket in S3. -
20
CubeFS
cloud-native file store
CubeFS is a new generation cloud-native storage that supports access protocols such as S3, HDFS, and POSIX. It is widely applicable in various scenarios such as big data, AI/LLMs, container platforms, separation of storage and computing for databases and middleware, data sharing and protection, etc. Compatible with various access protocols such as S3, POSIX, HDFS, etc., and the access between protocols can be interoperable. -
21
Amazon S3 PHP Class
A standalone Amazon S3 (REST) client for PHP 7.3+/cURL
A standalone Amazon S3 (REST) client for PHP 7.3+/cURL. -
22
AWS Service Provider for Laravel
A Laravel 5+ (and 4) service provider for the AWS SDK for PHP
...Get started quickly using AWS with the AWS SDK for PHP. The SDK is a modern, open-source PHP library that makes it easy to integrate your PHP application with AWS services like Amazon S3, Amazon Glacier, and Amazon DynamoDB. Conveniently marshal and unmarshal JSON documents and PHP arrays for storage in Amazon DynamoDB. Use PHP's built-in functions such as fopen, copy, and file_get_contents to work with objects stored in Amazon S3. Remove the session state scale bottleneck by easily storing application session state in DynamoDB. ... -
23
-
24
Zinc Search Engine
ZincSearch. A lightweight alternative to elasticsearch
...Docs coming soon. No need to work hard to define schema ahead of time. ZincSearch automatically discovers schema, so you can focus on search and analysis. Store data in S3 and MinIO for low cost, virtually infinite durable storage without the hassle of managing storage. -
25
Gobuster
Directory/File, DNS and VHost busting tool written in Go
...This project is born out of the necessity to have something that didn't have a fat Java GUI (console FTW), something that did not do recursive brute force, something that allowed me to brute force folders and multiple extensions at once, something that compiled to native on multiple platforms, something that was faster than an interpreted script (such as Python), and something that didn't require a runtime. Provides several modes, like the classic directory brute-forcing mode, DNS subdomain brute-forcing mode, the mode that enumerates open S3 buckets and looks for existence and bucket listings, and the virtual host brute-forcing mode (not the same as DNS!). Since this tool is written in Go you need to install the Go language/compiler/etc. Full details of installation and set up can be found on the Go language website. Once installed you have two options. You need at least go 1.16.0 to compile gobuster.
