AWS Neuron Files

Release Date: October 19, 2026

Today we are releasing Neuron 2.33.0. This release introduces TorchNeuron, the native PyTorch backend for AWS Trainium, now in public beta. The model code, libraries, and tools you already use run on Amazon EC2 Trn2 and Trn3 instances, in eager mode or with torch.compile. Neuron Observer, also now in public beta, detects issues in under a second and identifies the affected node or device.

In Neuron Kernel Interface (NKI) 0.7.0, kernel metaprogramming uses standard Python. The NKI Library adds 25 new experimental kernels for DeepSeek-V3.2, Qwen3.5, and Kimi Delta Attention. vLLM Neuron, launched in July 2026, is upgraded to vLLM 0.29.0 and uses native PyTorch compilation by default. vLLM Omni Neuron, launched in September 2026, is part of a Neuron release for the first time.

What’s in this release TorchNeuron (Public Beta) – Registers Trainium through PrivateUse1, PyTorch’s extension point for custom hardware, so the standard PyTorch APIs work as written on the neuron device. Both PyTorch execution modes run natively: eager for research and debugging, and torch.compile for production performance. The libraries you already use also run on Trainium, including TorchTitan, the Hugging Face libraries, and vLLM Neuron. Write custom kernels with NKI, and view torch.profiler traces in Neuron Explorer. See TorchNeuron: Native PyTorch on AWS Neuron.

Neuron Observer (Public Beta) – Monitors Trainium health from a single host to a cluster with a per-node daemon, the nobs command line client, and an on-demand health check. Collect metrics in Prometheus, Amazon CloudWatch, or Amazon Managed Service for Prometheus, and deploy the daemon on Amazon Elastic Kubernetes Service with a Helm chart. See Neuron Observer (Neuron 2.33.0 Release).

NKI 0.7.0 – Specializes kernels by tracing them with Python, so generators, class inheritance, and the rest of the language work in kernel metaprogramming. Tiled and blocked layouts address tensors by tile coordinate, and the CPU simulator runs fixed-size collective kernels across multiple ranks without Neuron hardware. See Neuron Kernel Interface (NKI) [0.7.0] (Neuron 2.33.0 Release).

NKI Library – Adds 25 new experimental kernels, including DeepSeek-V3.2 decode and prefill, Gated DeltaNet for Qwen3.5, Kimi Delta Attention, and a FlashAttention-2 forward kernel on Trn3. Each new kernel ships with a PyTorch reference implementation, and a new models/ folder holds kernels tuned for Gemma4 and GPT-OSS. See NKI Library (NKI-Lib) (Neuron 2.33.0 Release).

Graph Compiler – Reduces compile time for torch.compile models with heterogeneous architectures, such as vision-language, encoder-decoder, and mixture-of-experts models, with no code or flag changes. Rewritten backend analyses speed up large-graph compilation. On Trn3 under LNC=2 sharding, MXFP8 models keep quantized weights and MX matrix-multiplication outputs in on-chip memory, which lowers latency. See Neuron Compiler [2.28.7631.0] (Neuron 2.33.0 Release).

Runtime and Driver – New tensor cast and compute APIs run data-type conversion on the Neuron device, and element-wise addition, minimum, and maximum on Trn2 and Trn3. Large NEFFs load up to 6.7 times faster. Host-to-device and device-to-host transfers appear in profiles, variable-size all-to-all collectives span both racks of a Trn3 UltraServer, and the driver reports per-NeuronCore throttle levels. See Neuron Runtime (Neuron 2.33.0 Release).

Neuron Explorer – New HBM and PyTorch Allocator tabs in the Memory Viewer show device memory and PyTorch allocator timelines for system profiles. With a memory snapshot, the Memory Viewer attributes allocations to the Python source lines that made them, which helps debug out-of-memory errors in eager mode. Profiles also add host-to-device and device-to-host transfer tracks, each linked to the framework operation that triggered it, plus NKI call stacks and user-defined NKI regions. See Neuron Developer Tools & Neuron Explorer (Neuron 2.33.0 Release).

Neuron Agentic Development – The new neuron-parallelism-planner skill plans parallelism and batch size for large language model inference against latency targets, before you run on hardware. New skills also find the operator that limits a configuration, and localize bottlenecks in prefill and decode profiles captured from a running vLLM Neuron server. See Neuron Agentic Development (Neuron 2.33.0 Release).

vLLM Neuron – Moves to native PyTorch torch.compile as the default compilation path and improves inference performance: GPT-OSS-120B serves up to 2.5x more throughput than Neuron 2.32, and Qwen3-VL-32B up to 38% more, with faster server startup. Model Runner V2 is now the default for supported dense models. Adds Qwen3-32B dense model support and Qwen3-ASR-1.7B for speech-to-text. See vLLM Neuron [0.29.0.1.2.0] (Neuron 2.33.0 Release).

vLLM Omni Neuron – Moves to native PyTorch torch.compile and improves Wan2.2 text-to-video and image-to-video generation performance on Trn2 and Trn3. See vLLM Omni Neuron [0.29.0.0.2.0] (Neuron 2.33.0 Release).

JAX NeuronX – Updated for Neuron 2.33.0, with continued support for JAX 0.7.0–0.10.0. See JAX NeuronX (Neuron 2.33.0 Release).

DLAMIs and DLCs – Neuron Deep Learning AMIs (DLAMIs) add Ubuntu 26.04 in addition to Ubuntu 24.04, except the new vLLM Omni DLAMI, which is Ubuntu 24.04 only. Neuron Deep Learning Containers (DLCs) are published on Ubuntu 26.04 only. New torch-neuronx images provide PyTorch 2.14 with TorchTitan, and the DRA driver can add device taints that mark an individual Neuron device unavailable or degraded. See Neuron DLAMIs (Neuron 2.33.0 Release) and Neuron Containers (Neuron 2.33.0 Release).