AWS Neuron Files

Release Date: July 7, 2026

Today we are releasing Neuron 2.31.0. This release includes updates across the stack: NKI 0.5.0 — new MX FP8 scale dtype support, fewer instructions for indexed access patterns via tensor indirection, and zero-cost tensor layout transformations via new NkiTensor view APIs; 14 new NKI Library kernels for MoE training and DeepSeek MLA workloads; improved performance on Trn2 and Trn3 via a redesigned code generation backend in the Neuron Compiler; simplified Runtime configuration by eliminating the need to manually set scratchpad page sizes via contiguous shared scratchpad support; enhanced workload debugging in Neuron Explorer via System Trace Viewer source code linking and updated default grouping; and the UltraServer Operator — a Kubernetes operator for automated UltraServer discovery, workload allocation, and resource claim generation on Amazon EKS — now in public beta.

What’s in this release

• NKI 0.5.0 — Introduces MX FP8 scale dtype support, tensor indirection (gather/scatter) on compute operations, and new NkiTensor view APIs for zero-cost tensor layout transformations. Also adds expanded nc_matmul output capacity and IDE type stub support. See Neuron Kernel Interface (NKI) [0.5.0] (Neuron 2.31.0 Release).

• NKI Library — Adds 14 new experimental kernels covering deformable attention, MoE training collectives, indexed gather/scatter, DeepSeek MLA projection, and ring attention. Existing kernels gain a unified precision selector, FP8-packed KV paths, and PyTorch reference implementations. See NKI Library (NKI-Lib) (Neuron 2.31.0 Release).

• Graph Compiler — A redesigned code generation backend is now the default on Trn2 and Trn3, with improved instruction scheduling and memory prefetch for better runtime performance. Adds support for StableHLO composite operations including attention kernels. See Neuron Compiler (Neuron 2.31.0 Release).

• Runtime & Driver — Simplified device configuration with contiguous shared scratchpad support, improved collective communication with tensor list support and coalesced collective APIs, and enhanced debugging with out-of-bounds fault source reporting. The Driver adds live per-die power utilization monitoring, and Collectives adds multi-node ring communication on Trn2 and Trn3. See Neuron Runtime (Neuron 2.31.0 Release).

• Neuron Agentic Development — Updated NKI agentic skills for compatibility with NKI 0.5.0. See Neuron Agentic Development (Neuron 2.31.0 Release).

• Neuron Explorer — The System Trace Viewer adds source code linking and updated default grouping, and torch.compile and eager mode profiling are now supported. New supporting documentation includes a UI overview tour, environment variables reference, troubleshooting guide, glossary, and profile schema reference. See Neuron Developer Tools & Neuron Explorer (Neuron 2.31.0 Release).

• UltraServer Operator (Public Beta) — Automates UltraServer discovery, workload allocation, and resource claim generation for Trainium UltraServer workloads on Amazon EKS, eliminating the need for manual init containers, node label matching, and post-launch validation. See AWS Neuron UltraServer Operator.

• DLAMIs & Containers — All packages upgraded to Neuron SDK 2.31.0. See Neuron DLAMIs (Neuron 2.31.0 Release) and Neuron Containers (Neuron 2.31.0 Release).