Highlights

---

• Expanded the Python wheels into a C++ SDK, with linkable libraries for the CUDA, Core ML, MLX, OpenVINO, and Qualcomm delegates and packaged TorchAO kernels.
• Expanded LLM serving with multi-method export, batched request scheduling, bounded cancellation, off-graph KV-cache layouts, and new Qwen3.5 MoE, Muse Glimmer, Supertonic, and Voxtral workflows.
• Added ROCm support to the AOTI CUDA backend, including packed W4 execution and end-to-end Voxtral Realtime coverage.
• Expanded Arm and Cortex-M support with explicit-layout lowering and kernels, additional Ethos-U targets and demos, broader VGF profiling, and more model and operator coverage.
• Expanded Qualcomm LLM quantization-aware distillation, additional operators, Windows guidance, and stronger validation and error handling.
• Improved Vulkan quantized execution, dispatch safety, external-memory integration, and operator coverage.

Runtime and Core

---

• Added multi-method model support to the export pipeline and per-backend load options.
• Added capacity-aware memory planning over target memory maps and support for shared allocations with offsets.
• Added schema-version checks for PTE and PTD files and clearer failures for invalid device-planned copies and missing delegate data.
• Added shared pre-decomposition and graph-transform hooks, including common view/permute propagation and reinplace passes.
• Improved constant propagation, symbolic-shape handling, and device placement for memory-planned buffers.

Packaging and Build

---

• Added a C++ SDK to the Python wheel and split the runtime from the Python bindings extension.
• Added CUDA wheel publishing on Pytorch.org
• Improved delegate/runtime bundling in the published wheels.
• Added packaged TorchAO and quantized-kernel libraries.
• Added Windows ARM64 build coverage.
• Reduced wheel size and improved portability by stripping local macOS symbols, removing unreachable vendored sources, and dropping unusable Linux MKL search paths.
• Lowered the supported CMake floor so CMake 3.26 through 3.28 builds work again.

Backends

---

Arm

• Added opt-in Cortex-M explicit-layout lowering with convolution and pooling kernels.
• Added Arm export-recipe support, a pre-decomposition partitioner pipeline, and broader symbolic-shape and dim-order handling.
• Added Ethos-U65 support, Ethos-U85 KV-cache export, and SmolLM2, TinyStories-42M, Silero VAD, MobileSAM, and Stable Diffusion workflows.
• Expanded VGF profiling and environment preflight validation tools, neural statistics, automatic operator-support generation, support for int8 KV cache, FP16, BF16, MXFP8, and quantized model coverage.
• Improved Cortex-M memory planning, CMSIS-Pack coverage, in-place activation fusion, and arithmetic, pooling, and activation lowering.

Qualcomm

• Added LLM quantization-aware distillation and Gemma 4 text-model enablement.
• Added support for pdist, sort, reflection_pad3d, as_strided, scatter_reduce, scatter_add, and Hadamard-transform flows.
• Added QNN-on-Windows guidance and improved SDK setup, import behavior, delegate bounds checks, graph-I/O validation, and quantized metadata preservation.

CUDA and ROCm

• Added opt-in ROCm support to the AOTI backend, packed W4 execution, AMD pointwise examples, and Voxtral Realtime coverage on gfx1100 and gfx950.
• Added shared-memory targeting for cross-architecture CUDA inference and INT4, INT5, and INT6 short-query kernels.
• Improved CUDA delegate integration with per-thread streams, a warm memory pool, cross-method weight sharing, and clearer missing-weight errors.

Core ML and MLX

• Added linkable Core ML and MLX libraries to Apple frameworks and SwiftPM, plus ETDump profiling support.
• Added MLX off-graph KV-cache flat, ring, and cell layouts with shared pools and cross-thread execution fixes.
• Expanded MLX lowering with fused attention, native_group_norm, upsample_nearest2d, flip, and trunc support.
• Added Metal addmm and split_copy support and improved Core ML handling of tied embeddings and causal attention masks.

Vulkan

• Added external image and buffer wrapping with caller-provided dimensions.
• Added device-safe dispatch geometry, two-dimensional dispatch helpers, and quantized convolution paths using unsigned dot products.
• Added payload-bounded constant sharding, scalar comparisons, and broader arange, clamp, indexing, and batch support.

NXP

• Upgraded the eIQ Neutron SDK to 3.2.1 and added an MCUXpresso example build.
• Added Neutron support for rsqrt, Hardswish, softmax, reflected padding, scalar elementary arguments, and AdaptiveAvgPool1d.
• Added Neutron operator-suite and MLPerf Tiny classification coverage.

XNNPACK

• Added BF16 fully connected delegation and BF16 activation support in LLM Export.
• Bumped the XNNPACK pin to [92a7ad] to support the above
• Improved dynamic-quantized channels-last handling.
• Fixed pooling with default strides and single-element kernels, convolution dimensionality inference, and shared-observer handling for slices.

Operators and Kernels

---

• Added a quantized MoE feed-forward operator with INT4 packing, fused projections, source transformations, and numerical coverage.
• Added a quantized stacked-halves RoPE operator and BF16-by-FP32 GEMV decode support.
• Optimized BF16 SDPA and fixed NaNs in CUDA split-K attention for Voxtral.
• Added faster contiguous reductions and fixed non-contiguous reduction bounds, broadcast rank mismatches, and dim-order validation for several operators.

Notable Fixes

---

• Fixed CUDA low-memory external-weight export, model weight lookup, stream selection, and cross-method AOTI library collisions.
• Fixed QNN crashes on AMD hosts, invalid graph I/O, stale SDK bundles, and setup side effects during import.
• Fixed Arm convolution, pooling, rescale, grid-sampling, symbolic-shape, and quantization edge cases across TOSA, VGF, and Ethos-U.
• Fixed Vulkan delegation beyond supported graph regions and several dispatch, resize, and quantization synchronization issues.
• Fixed XNNPACK pooling, channels-last, placeholder-renaming, and BF16 export issues.
• Fixed Core ML and MLX attention, thread-affinity, and build-integration issues.

Contributors

We welcome 18 first-time contributors to ExecuTorch in this release: @denizkilinc, @yenhao-huang, @Kilbex, @ruanslv, @yuweix94, @slipstr34m, @AxelNoun, @adityasingh2400, @rstehle, @sid-rp, @malfet, @cthotti, @kkch3ng, @cgreenberg, @bharath-123, @ricardoasouz, @synath, @LuozhuZhang

Full Changelog

https://github.com/pytorch/executorch/compare/v1.4.0...v1.5.0