AWS Deep Learning Containers Integrations

Amazon Elastic Container Service (Amazon ECS) is a fully managed container orchestration service. Customers such as Duolingo, Samsung, GE, and Cook Pad use ECS to run their most sensitive and mission-critical applications because of its security, reliability, and scalability. ECS is a great choice to run containers for several reasons. First, you can choose to run your ECS clusters using AWS Fargate, which is serverless compute for containers. Fargate removes the need to provision and manage servers, lets you specify and pay for resources per application, and improves security through application isolation by design. Second, ECS is used extensively within Amazon to power services such as Amazon SageMaker, AWS Batch, Amazon Lex, and Amazon.com’s recommendation engine, ensuring ECS is tested extensively for security, reliability, and availability.

Whether you're looking for compute power, database storage, content delivery, or other functionality, AWS has the services to help you build sophisticated applications with increased flexibility, scalability and reliability. Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform, offering over 175 fully featured services from data centers globally. Millions of customers—including the fastest-growing startups, largest enterprises, and leading government agencies—are using AWS to lower costs, become more agile, and innovate faster. AWS has significantly more services, and more features within those services, than any other cloud provider–from infrastructure technologies like compute, storage, and databases–to emerging technologies, such as machine learning and artificial intelligence, data lakes and analytics, and Internet of Things. This makes it faster, easier, and more cost effective to move your existing applications to the cloud.

Amazon SageMaker is an advanced machine learning service that provides an integrated environment for building, training, and deploying machine learning (ML) models. It combines tools for model development, data processing, and AI capabilities in a unified studio, enabling users to collaborate and work faster. SageMaker supports various data sources, such as Amazon S3 data lakes and Amazon Redshift data warehouses, while ensuring enterprise security and governance through its built-in features. The service also offers tools for generative AI applications, making it easier for users to customize and scale AI use cases. SageMaker’s architecture simplifies the AI lifecycle, from data discovery to model deployment, providing a seamless experience for developers.

Amazon Elastic Compute Cloud (EC2) Trn1 instances, powered by AWS Trainium chips, are purpose-built for high-performance deep learning training of generative AI models, including large language models and latent diffusion models. Trn1 instances offer up to 50% cost-to-train savings over other comparable Amazon EC2 instances. You can use Trn1 instances to train 100B+ parameter DL and generative AI models across a broad set of applications, such as text summarization, code generation, question answering, image and video generation, recommendation, and fraud detection. The AWS Neuron SDK helps developers train models on AWS Trainium (and deploy models on the AWS Inferentia chips). It integrates natively with frameworks such as PyTorch and TensorFlow so that you can continue using your existing code and workflows to train models on Trn1 instances.

Amazon EC2 G5 instances are the latest generation of NVIDIA GPU-based instances that can be used for a wide range of graphics-intensive and machine-learning use cases. They deliver up to 3x better performance for graphics-intensive applications and machine learning inference and up to 3.3x higher performance for machine learning training compared to Amazon EC2 G4dn instances. Customers can use G5 instances for graphics-intensive applications such as remote workstations, video rendering, and gaming to produce high-fidelity graphics in real time. With G5 instances, machine learning customers get high-performance and cost-efficient infrastructure to train and deploy larger and more sophisticated models for natural language processing, computer vision, and recommender engine use cases. G5 instances deliver up to 3x higher graphics performance and up to 40% better price performance than G4dn instances. They have more ray tracing cores than any other GPU-based EC2 instance.

Amazon EC2 P4d instances deliver high performance for machine learning training and high-performance computing applications in the cloud. Powered by NVIDIA A100 Tensor Core GPUs, they offer industry-leading throughput and low-latency networking, supporting 400 Gbps instance networking. P4d instances provide up to 60% lower cost to train ML models, with an average of 2.5x better performance for deep learning models compared to previous-generation P3 and P3dn instances. Deployed in hyperscale clusters called Amazon EC2 UltraClusters, P4d instances combine high-performance computing, networking, and storage, enabling users to scale from a few to thousands of NVIDIA A100 GPUs based on project needs. Researchers, data scientists, and developers can utilize P4d instances to train ML models for use cases such as natural language processing, object detection and classification, and recommendation engines, as well as to run HPC applications like pharmaceutical discovery and more.

AWS Marketplace is a curated digital catalog that enables customers to discover, purchase, deploy, and manage third-party software, data products, and services directly within the AWS ecosystem. It provides access to thousands of listings across categories like security, machine learning, business applications, and DevOps tools. With flexible pricing models such as pay-as-you-go, annual subscriptions, and free trials, AWS Marketplace simplifies procurement and billing by integrating costs into a single AWS invoice. It also supports rapid deployment with pre-configured software that can be launched on AWS infrastructure. This streamlined approach allows businesses to accelerate innovation, reduce time-to-market, and maintain better control over software usage and costs.

Amazon Elastic Kubernetes Service (Amazon EKS) is a fully managed Kubernetes service. Customers such as Intel, Snap, Intuit, GoDaddy, and Autodesk trust EKS to run their most sensitive and mission-critical applications because of its security, reliability, and scalability. EKS is the best place to run Kubernetes for several reasons. First, you can choose to run your EKS clusters using AWS Fargate, which is serverless compute for containers. Fargate removes the need to provision and manage servers, lets you specify and pay for resources per application, and improves security through application isolation by design. Second, EKS is deeply integrated with services such as Amazon CloudWatch, Auto Scaling Groups, AWS Identity and Access Management (IAM), and Amazon Virtual Private Cloud (VPC), providing you a seamless experience to monitor, scale, and load-balance your applications.

Easily store, share, and deploy your container software anywhere. Push container images to Amazon ECR without installing or scaling infrastructure, and pull images using any management tool. Share and download images securely over Hypertext Transfer Protocol Secure (HTTPS) with automatic encryption and access controls. Access and distribute your images faster, reduce download times, and improve availability using a scalable, durable architecture. Amazon ECR is a fully managed container registry offering high-performance hosting, so you can reliably deploy application images and artifacts anywhere. Meet your organization’s image compliance security requirements using insights from common vulnerabilities and exposures (CVEs) and the Common Vulnerability Scoring System (CVSS). Publish containerized applications with a single command and easily integrate your self-managed environments.

It supports high-performance training on AWS Trainium-based Amazon Elastic Compute Cloud (Amazon EC2) Trn1 instances. For model deployment, it supports high-performance and low-latency inference on AWS Inferentia-based Amazon EC2 Inf1 instances and AWS Inferentia2-based Amazon EC2 Inf2 instances. With Neuron, you can use popular frameworks, such as TensorFlow and PyTorch, and optimally train and deploy machine learning (ML) models on Amazon EC2 Trn1, Inf1, and Inf2 instances with minimal code changes and without tie-in to vendor-specific solutions. AWS Neuron SDK, which supports Inferentia and Trainium accelerators, is natively integrated with PyTorch and TensorFlow. This integration ensures that you can continue using your existing workflows in these popular frameworks and get started with only a few lines of code changes. For distributed model training, the Neuron SDK supports libraries, such as Megatron-LM and PyTorch Fully Sharded Data Parallel (FSDP).

Amazon Elastic Compute Cloud (Amazon EC2) P5 instances, powered by NVIDIA H100 Tensor Core GPUs, and P5e and P5en instances powered by NVIDIA H200 Tensor Core GPUs deliver the highest performance in Amazon EC2 for deep learning and high-performance computing applications. They help you accelerate your time to solution by up to 4x compared to previous-generation GPU-based EC2 instances, and reduce the cost to train ML models by up to 40%. These instances help you iterate on your solutions at a faster pace and get to market more quickly. You can use P5, P5e, and P5en instances for training and deploying increasingly complex large language models and diffusion models powering the most demanding generative artificial intelligence applications. These applications include question-answering, code generation, video and image generation, and speech recognition. You can also use these instances to deploy demanding HPC applications at scale for pharmaceutical discovery.