AWS Neuron Files

AWS Neuron SDK 2.27.0 Release Notes

This release adds support for Trainium3 (Trn3) instances. Enhanced NKI with new NKI Compiler introduces the nki.* namespace with updated APIs and language constructs. The NKI Library provides pre-optimized kernels for common model operations including attention, MLP, and normalization. Neuron Explorer delivers a unified profiling suite with AI-driven optimization recommendations. vLLM V1 integration is now available through the vLLM-Neuron Plugin. Deep Learning Containers and AMIs are updated with vLLM V1, PyTorch 2.9, JAX 0.7, Ubuntu 24.04, and Python 3.12.

In addition to this release, we are introducing new capabilities and features in private beta access (see Private Beta Access section). We are also announcing our transition to PyTorch native support starting with PyTorch 2.10 in Neuron 2.28, plans to simplify NxDI in upcoming releases, and other important updates. See the End of Support and Migration Notices section for more details.

Neuron Kernel Interface (NKI)

NKI Compiler - The new nki.* namespace replaces the legacy neuronxcc.nki.* namespace. Top-level kernel functions now require the @nki.jit annotation. Neuron 2.27 supports both namespaces side by side; the legacy namespace will be removed in Neuron 2.28. A kernel migration guide is available in the documentation.

NKI Library

The NKI Library provides pre-optimized kernels: Attention CTE, Attention TKG, MLP, Output Projection CTE, Output Projection TKG, QKV, and RMSNorm-Quant. Kernels are accessible via the nkilib.* namespace in neuronx-cc or from the GitHub repository.

Developer Tools

Neuron Explorer - A suite of tools designed to support ML engineers throughout their development journey on AWS Trainium. This release features improved performance and user experience for device profiling, with four core viewers to provide insights into model performance:

• Hierarchy Viewer: Visualizes model structure and component interactions
• AI Recommendation Viewer: Delivers AI-driven optimization recommendations
• Source Code Viewer: Links profiling data directly to source code
• Summary Viewer: Displays high-level performance metrics

Neuron Explorer is available through UI, CLI, and VSCode IDE integration. Existing NTFF files are compatible but require reprocessing for new features.

New tutorials cover profiling NKI kernels, multi-node training jobs, and vLLM inference workloads. The nccom-test tool now includes fine-grained collective communication support.

Inference Updates

vLLM V1 - The vLLM-Neuron Plugin enables vLLM V1 integration for inference workloads. vLLM V0 support ends in Neuron 2.28.

NxD Inference - Model support expands with beta releases of Qwen3 MoE (Qwen3-235B-A22B) for multilingual text and Pixtral (Pixtral-Large-Instruct-2411) for image understanding. Both models use HuggingFace checkpoints and are supported on Trn2 and Trn3 instances.

Neuron Graph Compiler

Default accuracy settings are now optimized for precision. The --auto-cast flag defaults to none (previously matmul), and --enable-mixed-precision-accumulation is enabled by default. FP32 models may see performance impacts; restore previous behavior with --auto-cast=matmul and --disable-mixed-precision-accumulation. Python 3.10 or higher is now required.[]

Runtime Improvements

Neuron Runtime Library 2.29 adds support for Trainium3 (Trn3) instances and delivers performance improvements for Collectives Engine overhead, NeuronCore branch overhead, NEFF program startup, and all-gather latency.

Deep Learning AMIs and Containers

Platform Updates - All DLCs are updated to Ubuntu 24.04 and Python 3.12. DLAMIs add Ubuntu 24.04 support for base, single framework, and multi-framework configurations.

Framework Updates:

• vLLM V1 single framework DLAMI and multi-framework virtual environments
• PyTorch 2.9 single framework DLAMIs and multi-framework virtual environments (Amazon Linux 2023, Ubuntu 22.04, Ubuntu 24.04)
• JAX 0.7 single framework DLAMI and multi-framework virtual environments

New Container - The pytorch-inference-vllm-neuronx 0.11.0 DLC provides a complete vLLM inference environment with PyTorch 2.8 and all dependencies.

Read What's New: Neuron 2.27.0 and Neuron 2.27.0 component release notes for specific Neuron component improvements and details.