Search Results for "s3"
Sort By:
Showing 38 open source projects for "s3"
View related business solutions-
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
s3-client
Sample python script to work with Amazon S3
Example Python script to work with S3. -
2
AWS Data Wrangler
Pandas on AWS, easy integration with Athena, Glue, Redshift, etc.
...Easy integration with Athena, Glue, Redshift, Timestream, OpenSearch, Neptune, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON, and EXCEL). Built on top of other open-source projects like Pandas, Apache Arrow and Boto3, it offers abstracted functions to execute usual ETL tasks like load/unload data from Data Lakes, Data Warehouses, and Databases. Convert the column name to be compatible with Amazon Athena and the AWS Glue Catalog. Run a query against AWS CloudWatchLogs Insights and convert the results to Pandas DataFrame. ... -
3
Ansible Role s3cmd
Ansible role for s3cmd. Available on Ansible Galaxy
Role to install (by default) s3cmd on Debian/Ubuntu and EL systems. s3cmd is a popular s3 client. -
4
SageMaker TensorFlow Training Toolkit
Toolkit for running TensorFlow training scripts on SageMaker
...After creating a SageMaker Model, you can use it to create SageMaker Batch Transform Jobs for offline inference, or create SageMaker Endpoints for real-time inference. A SageMaker Model contains references to a model.tar.gz file in S3 containing serialized model data, and a Docker image used to serve predictions with that model. A Batch Transform job runs an offline-inference job using your TensorFlow Serving model. Input data in S3 is converted to HTTP requests, and responses are saved to an output bucket in S3. -
Enterprise-grade ITSM, for every businessFreshservice is an intuitive, AI-powered platform that helps IT, operations, and business teams deliver exceptional service without the usual complexity. Automate repetitive tasks, resolve issues faster, and provide seamless support across the organization. From managing incidents and assets to driving smarter decisions, Freshservice makes it easy to stay efficient and scale with confidence.
-
5
django-dbbackup
Management commands to help backup and restore your project database
django-dbbackup is a Django management command extension for backing up and restoring databases and media files. It supports multiple storage backends including local file systems, Amazon S3, and Dropbox. Ideal for automated backup strategies, it simplifies the integration of backup logic into Django projects. -
6
Cookiecutter Django
Framework for jumpstarting production-ready Django projects quickly
...Provides an optional basic ASGI setup for Websockets and an optional custom static build using Gulp and livereload. Send emails via Anymail (using Mailgun by default or Amazon SES if AWS is selected cloud provider, but switchable). Media storage using Amazon S3 or Google Cloud Storage. Docker support using docker-compose for development and production (using Traefik with LetsEncrypt support). Procfile for deploying to Heroku. Provides instructions for deploying to PythonAnywhere. You can run tests with unittest or pytest. -
7
DVC
Data Version Control | Git for Data & Models
...It is designed to handle large files, data sets, machine learning models, and metrics as well as code. Version control machine learning models, data sets and intermediate files. DVC connects them with code and uses Amazon S3, Microsoft Azure Blob Storage, Google Drive, Google Cloud Storage, Aliyun OSS, SSH/SFTP, HDFS, HTTP, network-attached storage, or disc to store file contents. Version control machine learning models, data sets, and intermediate files. DVC connects them with code and uses Amazon S3, Microsoft Azure Blob Storage, Google Drive, Google Cloud Storage, Aliyun OSS, SSH/SFTP, HDFS, HTTP, network-attached storage, or disc to store file contents. ... -
8
CTFd
CTFs as you need them
...Challenge plugin architecture to create your own custom challenges. Static & Regex-based flags. Custom flag plugins. Unlockable hints. File uploads to the server or an Amazon S3-compatible backend. Limit challenge attempts & hide challenges. Automatic bruteforce protection. Individual and Team-based competitions. Have users play on their own or form teams to play together. Scoreboard with automatic tie resolution. Hide Scores from the public. Freeze Scores at a specific time. Scoregraphs comparing the top 10 teams and team progress graphs. ... -
9
Amazon Braket Python Schemas
A library that contains schemas for Amazon Braket
Amazon Braket Python Schemas is an open source library that contains the schemas for Braket, including intermediate representations (IR) for Amazon Braket quantum tasks and offers serialization and deserialization of those IR payloads. Think of the IR as the contract between the Amazon Braket SDK and Amazon Braket API for quantum programs. Schemas for the S3 results of each quantum task. Schemas for the device capabilities of each device. The preferred way to get Amazon Braket Python Schemas is by installing the Amazon Braket Python SDK, which will pull in the schemas. You can install from source by cloning this repository and running a pip install command in the root directory of the repository. ... -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
10
Agent S
Agent S: an open agentic framework that uses computers like a human
...Built to operate graphical user interfaces like a human, it allows AI agents to perceive screens, reason about tasks, and execute actions across macOS, Windows, and Linux systems. The latest version, Agent S3, surpasses human-level performance on the OSWorld benchmark, demonstrating state-of-the-art results in complex multi-step computer tasks. Agent S combines powerful foundation models (such as GPT-5) with grounding models like UI-TARS to translate visual inputs into precise executable actions. It supports flexible deployment via CLI, SDK, or cloud, and integrates with multiple model providers including OpenAI, Anthropic, Gemini, Azure, and Hugging Face endpoints. ... -
11
AWS SDK for pandas
Easy integration with Athena, Glue, Redshift, Timestream, Neptune
aws-sdk-pandas (formerly AWS Data Wrangler) bridges pandas with the AWS analytics stack so DataFrames flow seamlessly to and from cloud services. With a few lines of code, you can read from and write to Amazon S3 in Parquet/CSV/JSON/ORC, register tables in the AWS Glue Data Catalog, and query with Amazon Athena directly into pandas. The library abstracts efficient patterns like partitioning, compression, and vectorized I/O so you get performant data lake operations without hand-rolling boilerplate. It also supports Redshift, OpenSearch, and other services, enabling ETL tasks that blend SQL engines and Python transformations. ... -
12
AWS Chalice
Python Serverless Microframework for AWS
...You define routes, event handlers, and background tasks in plain Python, and Chalice turns them into AWS Lambda functions wired to API Gateway, Amazon EventBridge schedulers, S3/SNS/SQS triggers, and more. A single command builds your app, bundles dependencies, generates infrastructure templates, and deploys to your account with sensible defaults. The framework includes local development tools so you can iterate on HTTP routes and inspect logs before shipping to the cloud. It also manages per-stage configuration, environment variables, IAM permissions scaffolding, and packaging optimizations to keep cold starts small. ... -
13
Deep Lake
Data Lake for Deep Learning. Build, manage, and query datasets
...It can be deployed locally or in the cloud, and it enables you to store all of your data in one place, ranging from simple annotations to large videos. Deep Lake is used by Google, Waymo, Red Cross, Omdena, Yale, & Oxford. Use one API to upload, download, and stream datasets to/from AWS S3/S3-compatible storage, GCP, Activeloop cloud, or local storage. Store images, audios and videos in their native compression. Deeplake automatically decompresses them to raw data only when needed, e.g., when training a model. Treat your cloud datasets as if they are a collection of NumPy arrays in your system's memory. Slice them, index them, or iterate through them. -
14
PGHoard
PostgreSQL® backup and restore service
pghoard is a PostgreSQL backup and restore tool that provides encrypted, compressed, and cloud-optimized backups. Developed by Aiven, it supports streaming WAL archiving and full base backups to various cloud storage backends. pghoard is designed for reliability and fast disaster recovery in cloud-native PostgreSQL deployments. -
15
aws-devops-zero-to-hero
AWS zero to hero repo for devops engineers to learn AWS in 30 Days
...The README is structured as a day-by-day syllabus, starting with “Day 1: Introduction to AWS” and moving through IAM, EC2, VPC networking, security, DNS (Route 53), storage (S3), and many other core services. Each day mixes explanation with at least one concrete project or lab, such as deploying applications on EC2, designing secure VPCs, setting up CI/CD pipelines, or configuring CloudWatch monitoring. Later in the curriculum, you move into topics like CloudFormation, CodeCommit/CodePipeline/CodeBuild/CodeDeploy, Terraform on AWS, CloudTrail and Config for compliance, Elastic Load Balancing, and cloud migration strategies. ... -
16
Scrapy
A fast, high-level web crawling and web scraping framework
Scrapy is a fast, open source, high-level framework for crawling websites and extracting structured data from these websites. Portable and written in Python, it can run on Windows, Linux, macOS and BSD. Scrapy is powerful, fast and simple, and also easily extensible. Simply write the rules to extract the data, and add new functionality if you wish without having to touch the core. Scrapy does the rest, and can be used in a number of applications. It can be used for data mining, monitoring... -
17
aws-cli
Universal Command Line Interface for Amazon Web Services
The AWS CLI is the universal command-line interface for managing AWS services, automating tasks, and scripting cloud workflows. It exposes nearly every public API from EC2 and S3 to IAM, Lambda, and beyond, providing parity with the service SDKs in a tool you can run anywhere. Profiles, regions, single-sign-on, and credential helpers make it straightforward to switch contexts securely across accounts and environments. Its output controls and JMESPath querying let you slice, filter, and transform JSON responses directly in the shell, which is essential for automation. ... -
18
cloud_enum
Multi-cloud OSINT tool for discovering public cloud resources
cloud_enum is an open source reconnaissance and OSINT tool designed to discover publicly accessible cloud resources across major cloud providers. It focuses on enumerating assets in Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform using keyword-based discovery techniques. It works by taking user-provided keywords and generating variations through mutation wordlists, then testing these combinations against common cloud service naming patterns. cloud_enum performs both... -
19
Kedro
A Python framework for creating reproducible, maintainable code
Kedro is an open sourced Python framework for creating maintainable and modular data science code. Provides the scaffolding to build more complex data and machine-learning pipelines. In addition, there's a focus on spending less time on the tedious "plumbing" required to maintain data science code; this means that you have more time to solve new problems. Standardises team workflows; the modular structure of Kedro facilitates a higher level of collaboration when teams solve problems... -
20
SQL Explorer
Easily share data across your company via SQL queries
SQL Explorer aims to make the flow of data between people fast, simple, and confusion-free. It is a Django-based application that you can add to an existing Django site, or use as a standalone business intelligence tool. Quickly write and share SQL queries in a simple, usable SQL editor, preview the results in the browser, share links, download CSV, JSON, or Excel files (and even expose queries as API endpoints, if desired), and keep the information flowing! Comes with support for multiple... -
21
AWS Agent Skills
AWS Skills for Agents
...Instead of streaming giant documentation sets or relying on episodic web search, this project compresses AWS best practices, usage patterns, edge cases, and real-world engineering guides into pre-structured skill definitions that are token-efficient and tailored for reasoning. The skills cover critical AWS services such as IAM, Lambda, DynamoDB, S3, API Gateway, EKS, and many more, letting agents offer actionable advice on infrastructure as code, debugging, security configurations, and architectural workflows. Skills are kept up to date with weekly documentation checks, ensuring they reflect current AWS patterns and service changes. -
22
SDGym
Benchmarking synthetic data generation methods
The Synthetic Data Gym (SDGym) is a benchmarking framework for modeling and generating synthetic data. Measure performance and memory usage across different synthetic data modeling techniques – classical statistics, deep learning and more! The SDGym library integrates with the Synthetic Data Vault ecosystem. You can use any of its synthesizers, datasets or metrics for benchmarking. You also customize the process to include your own work. Select any of the publicly available datasets from the... -
23
AWS IoT Device SDK for Python
SDK for connecting to AWS IoT from a device using Python
...By connecting their devices to AWS IoT, users can securely work with the message broker, rules, and the device shadow (sometimes referred to as a thing shadow) provided by AWS IoT and with other AWS services like AWS Lambda, Amazon Kinesis, Amazon S3, and more. It is a complete rework, built to improve reliability, performance, and security. We invite your feedback! The SDK is built on top of a modified Paho MQTT Python client library. Developers can choose from two types of connections to connect to AWS IoT. For MQTT over TLS (port 8883 and port 443), a valid certificate and a private key are required for authentication. ... -
24
ConsoleMe
A central control plane for AWS permissions and access
...Weep (ConsoleMe’s CLI) supports 5 different ways of serving AWS credentials locally. Cloud administrators can create/clone IAM roles and natively manage IAM roles, users, inline/managed policies, S3 Buckets, SQS queues, and SNS topics across hundreds of accounts in a single interface. Users can access most of your cloud resources in the AWS Console with a single click. Cloud administrators can configure ConsoleMe to authenticate users through ALB Authentication, OIDC/OAuth2, or SAML. -
25
S3cmd
Command line tool for managing Amazon S3 and CloudFront services
S3cmd (s3cmd) is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage or DreamHost DreamObjects. It is best suited for power users who are familiar with command-line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc. S3cmd is written in Python. It's an open-source project available under GNU Public License v2 (GPLv2) and is free for both commercial and private use. ...
