whisper.cpp Files

Overview

New version has been released.

Nightly build: b5130 More info: dist : releases and versioning of ggml-org projects

Changelog since v1.9.3

927cfce3 metal : remove leftover ggml-metal.metal kernels file (#4051) dd80eb2a scripts : use sed instead of grep for version parsing [no ci] (#4052) 1fa6dfa0 ci : update WoA CUDA 13.4 to use 13.4.1 GA (#4053) a2b36eb6 whisper : bump version to 1.9.4 (#4050) 6fb4cd67 ci : add Windows On ARM support to release job (#4048) c44b60b8 whisper : call encoder_begin_callback before language auto-detect (#3936) 6d0ed914 whisper : re-seed decoder 0 between calls (#4025) cec4dbe5 ci : update close-issue job to not close issues (#4045) 79f2d921 tests : load backends before init when built with GGML_BACKEND_DL (#4031) 61e6ccad server : return language in detect response (#4035) 52a939a2 sync : ggml a937f4e8 ggml : bump version to 0.23.0 (ggml/1618) 11d4eec8 metal : add remaining fa-vec tunings for M3 Max (llama/28373) 140e57a4 ggml : replace compile definitions with version.h.in (llama/28364) e2389eb9 ggml : rename and make private ggml_op_alloc_size_may_expand() (ggml/0) 1b37bead ggml : don't crash when backend search path can't be read (llama/28271) f32e6fa0 ggml : remove GGML_CUDA_PEER_MAX_BATCH_SIZE (llama/28177) e1bbe405 ggml-cpu(s390x) : fix q5_1 uninitialized v_acc (llama/28332) d1e0e649 sycl: fuse rms_norm+mul+add and add+add residual chains (llama/27610) 36f170e5 SYCL: Refactor GGML_SYCL_ENABLE_MKL_FA to global var (llama/26863) d784add7 opencl: quant lm_head / decode GEMV and medium-batch GEMM optimizations (speculative decoding/MTP) (llama/26477) 0a4a95c8 tune MMVQ to MMQ crossover for SM87 (llama/28285) 4dd48dde metal : add sparse FA (llama/28098) d55d345e metal : fix glu dispatch with ne00 = 1 (llama/28306) 25350b57 CUDA: Allow concurrent streams per split for multi-GPU (llama/28198) 47d348a2 vulkan: fix FA dequant path engagement (llama/28190) f24a3860 sycl : enhance the api to support peer-to-peer copy (llama/27550) a704770e sycl: reduce redundant work in Q4_K multi-column MMVQ (llama/27062) e5605697 finetune: fix no KV cache (llama/27199) 37f0f443 ggml-hexagon: add F16 support for unary ops (llama/28228) 1bdda1e3 metal : add fa-vec tunings for M3 (llama/28236) 3a1c7d6b metal : fix memory query under low-memory conditions (llama/27701) 4d343d7c ggml-cuda : remove unused vars (llama/28235) 519df618 CUDA + ggml: add sparse-fa for DSV4/GLM (llama/27970) c2b40075 ggml: avoid KleidiAI buffer type init on dispatch (llama/27891) dc70853e hexagon: MUL_MAT and MUL_MAT_ID fusion and fixes (llama/28202) 1c7d35e1 vulkan: handle larger batch sizes (>4) efficiently for IQ3_S mat-vec (llama/27449) a9e58612 vulkan : only request VK_KHR_shader_bfloat16 extension if supported (llama/28155) d57ae982 ggml-cpu : conditionally add SpacemiT IME kernel sources (llama/27961) 35133c94 opencl: fix out‐of‐bound reads in the Adreno image kernels (#27632) c94921f8 hexagon: add missing FARF logs for cpy/get_rows/set_rows/gdn ops (llama/28217) fcc2feee metal : add metallib build support for xcframework (llama/28163) 2c486783 cuda: fuse MoE weighted expert reduction (llama/25952) f162a194 Revert "sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 12…" (#28184) 408faaaf sycl : add Kronecker product FWHT support for sizes 384, 640, 768, 1280 (llama/28016) 5f07f856 metal : add fa-vec tuning for M2 Pro (llama/28122) 8cca1a36 metal : add fa-vec tunings for A18 Pro (MacBook Neo) (llama/28152) a245a8f4 metal : fix more leaks due to missing autoreleasepools (llama/27883) 870db2af metal : add fa-vec tuning for M2 Max (llama/28015) 4f3a2a4b sycl : support limit max alloc memory within 2GB for host-pinned memory (llama/27559) 5032008b metal: enable Metal 4.0 tensor API on M5+/A19+ (llama/27461) 8e54c659 metal : add fa-vec tunings for M1 Ultra (llama/28088) f22bb2ea CUDA: XOR swizzle flash attn K,V smem fp16 tiles (llama/25635) dbc40efc metal : add concat support for quantized types (llama/28116) 2f608ab4 AVX2: Speed up large batch size prompt processing of IQ models (llama/27402) c6934d0f metal : add top-k radix implementation (llama/28073) 088c603e opencl: tune the quant paths for Intel Xe-LP GPUs to improve its TG and PP performance (llama/26438) c648b9a4 webgpu : avoid crash when offset is not multiple of 4 in WebGPU ggml_backend_tensor_get() implementation (llama/28045) c1be45b8 ROCm: add radix TOP_K for long rows (llama/27466) 7614a4c1 metal : add fa-vec tunings for M1 (llama/28078) b0f4bc02 CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token (llama/27621) 76a51e82 sycl : Enhance to get the free memory of Intel GPU (llama/27968) 6ce7b899 vulkan: tune mat-vec rows for batched inference on Strix Halo (llama/27909) 96dddd87 ggml : add MUL_MAT to the list of ops that may need additional memory (for WebGPU) (llama/28071) db00b019 vulkan: top_k radix select for k >= 1024 for Qwen 3.8 Flash Next (llama/28032) 01ebd225 hexagon: fix CPY fence bug (llama/28033) e5c96ca4 metal : add remaining Q4_1/Q5_0/Q5_1 fa-vec tunings for M2 (llama/28017) 4089fa62 rpc: avoid serializing buffers from other servers (llama/26500) 749683d3 ggml : fix ggml_backend_buft_get_alloc_size() guard (llama/28038) e9583f07 ggml: add SWIGLU_CLAMP (llama/27930) e900a732 CUDA: use the fast mm_ids_helper path for any n_expert_used (llama/27978) 35d9e223 hip: tune rdna 3 mmq config (llama/26284) e5c9e3e3 hip : optimize Q2_0 dot-product path for gfx1201 (llama/26753) 4b2243a6 ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (llama/27960) 43acf3d6 rpc: fix apple rdma error spew on teardown (llama/27908) 1e0f3825 metal: add fa-vec tunings for M3 Ultra (llama/27999) b66593ef metal : Add fa-vec tuning for M3 Pro (llama/27963) 5e494599 rpc : fix pre-rdma macOS versions (llama/27815) 3ad8b9b2 hexagon: support for device discovery and create sessions on demand (llama/27785) 3d4e0e98 sycl: split long rows in TOP_K instead of one work-group per row (llama/27847) c68f2055 metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (llama/25648) c969c68b ggml: allow passing alloc dependencies in graph_optimize (llama/27301) b33bbc5c metal : add fa-vec tunings for M2 (llama/27940) 2a11026c opencl: use a better matmul path on two Adreno GPU generations (llama/27640) 285f1ffd metal : assert shared memory padding (llama/27951) 308fa4f8 metal : add remaining fa-vec tunings for M4 Pro (llama/27915) 325c8d16 sycl: make --fit respect --fit-target better (llama/27629) e6447520 vulkan: combine duplicated fastdiv functions, rename the one optimizing small divs (llama/27526) 590fe189 metal : add fa-vec tunings for M1 Max (llama/27932) d501a0a3 vulkan: Change mul_mat_id to pad K rather than N (llama/27925) 4c38040f vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (llama/27812) caea96f6 ggml : fix conv_transpose_2d for multiple batches (llama/26132) ba99c098 Vulkan: add hoisting support for row IDs and expert count in shaders (llama/26686) 0a150873 metal : add fa-vec tunings for M4 (llama/27875) 7f78e1b4 OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops (llama/27843) fa4d244c sycl: use TILE for quantized KV decode on BMG (llama/26689) 97d0da26 sycl: bind the f16 KV cache in place for the oneDNN SDPA path (llama/27468) 530e3f48 metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (llama/27863) ff38b98e metal : add fa-vec tunings for M4 Pro (llama/27824) b6571e4a ggml-hexagon: add HTP unary ops for ABS and LOG (llama/27786) a0614d9e hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast norms (llama/27798) 85299715 opencl: add bin kernels kernel_gemm_moe_q4_0_q8_1_dp4a_bin, kernel_gemm_moe_mxfp4_q8_1_dp4a_bin (llama/27768) 55ab1e51 Feature: Added LIGHTNING_INDEXER support for Deepseek V4 ops on Vulkan Backend (llama/27453) a5db1d66 metal : fix memory leaks due to missing autoreleasepools (llama/27758) 3fea10db hexagon: support for multi-NPU devices (IQ9, IQ10) and fully asynchronous backend (llama/26501) 5271734e vulkan: warptiles currently assume warp sizes <= 64, clamp to work around larger warps (llama/27726) 0a026972 Implemented vulkan cross_entropy_loss and cross_entropy_loss_back (llama/27216) 82f5f85e rpc : implement event and async backend APIs (llama/18626) 9d8e6b91 cuda: unblock mmq for MoE on sm_60 (llama/26264) 8c0adb05 ggml-metal: add chunked SSD MMA for Mamba-2 prefill optimization (llama/26647) 8df657a2 ggml-meta: propagate buffer usage and call init on the new tensors (llama/27586) 482956e7 kleidiai: Rework KleidiAI Build System/Integration (llama/26077) e820c280 rpc: support apple RDMA as an RPC transport (llama/26421) be12d39a metal : null-check buffer alloc to fix OOM crash (llama/25371) 642b5d32 ruby : Add #free method, check MemoryView strictly (#4032) eacbd823 whisper : default-initialize whisper_mel to avoid uninitialized read (#3981) c4ac0012 parakeet : fix TDT decode by outputting raw logits from the joint graph (#4017) 97811330 talk-llama : sync llama.cpp 3680f66f pi : init 0414519d sync : ggml d470c9d4 ggml : bump version to 0.22.0 (ggml/1607) 322a77cf sycl : mark tq2_0 as not supported (llama/27660) 17a522a8 webgpu : fix handling of infinity values during ARGSORT and TOP_K (llama/27538) fa3b87c1 metal : per-device tuned (Q, NE) for flash-attn vec (llama/26570) fe52277c sync : ggml b15d31d0 metal: per-op source split + parallel compile (llama/26561) 42574451 scripts : update ggml-am 1d8e0529 sync : ggml aa25d33f ggml : shorten virtual device naming in CUDA and Metal (llama/27608) 103305e6 webgpu : reorder includes since V that appears in common_decls.tmpl may be defined as K in flash_attn_decls.tmpl if KV_OVERLAP (llama/27545) 32d0f49d ggml : fix ggml_clamp (llama/27644) 20209c25 Deepseek 4: -sm tensor (llama/26490) 3b89b37c Fix meta tensor split state propagation (llama/27574) c8a40099 cuda : add POOL_1D support (llama/27573) 13a7856d vulkan : added the PAD_REFLECT_1D operation (llama/26586) 1efb31e6 ggml: optimize concat op by replacing per-element memcpy with row-level memcpy (llama/24575) 21a67dd8 sycl : add Q2_K reordered MMVQ and ESIMD kernels (again) (llama/27490) 5f7bd9dd opencl: fold the gpt-oss MoE per-expert bias adds into the epilogue (op/kernel fusion) (llama/26431) a722846c whisper : guard null source in buffer loader read callback (#3982) c122757f docs : center badges in README.md [no ci] (#4012) 25694098 devops : add main-rocm Dockerfile (#3975) 52dec9d8 vitisai : add VitisAI Plugin for AMD Ryzen AI NPU encoder offload (#3608) 233fe1fc whisper : bypass cross-attention scaling for OpenVINO backend (#3997) 51de5e8b openvino : update model conversion and README.md (#4003) 3391d6b6 scripts : add release.sh script (#4010) a4610c78 docs : add release badge and remove stable/roadmap [no ci] (#4009) ab578879 make : add --parallel to cmake build command (#4007) 45f1593f sync : ggml ce77728c Revert "sycl : add Q2_K reordered MMVQ and ESIMD kernels (llama/26336)" (llama/27486) 0d9ba28c ggml : bump version to 0.21.0 (ggml/1597) d6c416e2 kleidiai : add SME2 F32 GEMV kernel support (llama/26891) d60ef650 sycl : add Q2_K reordered MMVQ and ESIMD kernels (llama/26336) b1cb8059 sycl : Add Q5_K ESIMD kernel (llama/26376) 2cb52ddb opencl: keep the vocab-scale K-quant lm_head on the CPU for Adreno A7X (compiler issue workaround) (llama/26440) af74f979 sycl: Update gate logic for Alchemist GPUs regarding OneDNN features. (llama/26635) f19250e5 sycl: fix multiple warnings in compiling sycl backend (llama/26713) 12137c31 sycl : fix load model with mlock issue (llama/27250) 5656e44e ggml: support ggml_rope_set_offset on opencl, sycl, wgpu, hexagon (llama/27345) d68216aa metal : clamp K extent in tensor API mat-mat kernel for K not a multiple of 32 (llama/27450) c16cb42f opencl: fix q6_K flat mul_mat for Adreno A6x/A7x GPUs with older E031 compilers (llama/26476) 73c2b7ec opencl: fix local size for norm (llama/27339) 8539d72f vulkan: FA MMQ should use fp32 for Q quantization calculations (llama/27413) 60f14a5e metal : dequant kv cache only for large batches (llama/27438) 29224181 CI: Use LLVM's OpenMP over MSVC_DEBUG_non_redist on Windows (llama/26678) c2dc946f CUDA: adding switch points per HW and quant type to tune the mvq->MMQ decode crossover (llama/26079) 283775e0 metal : dequantize quantized KV to F16 before flash attention (llama/27390) 02be8f5d Revert "tensor-split meta backend fixes (ggml/26502)" (llama/27433) 13fa8607 ggml: fix backend split scheduler race condition (llama/26040) 2648a706 ggml-cuda: provide static workspace for cuBLAS handles (llama/26574) acfad328 vulkan : add source groups for shaders (llama/26666) a3386d40 opencl: make the MoE expert scatter deterministic (llama/26464) 135f348b tensor-split meta backend fixes (llama/26502) 9f4b18ae hexagon: fix FA HMX queue ordering and pack the rescale D matrices (llama/27042) cd340ea8 opencl: port fused ssm_scan kernel (Mamba-2, d_state in {128, 256}) to GPU (llama/26439) 3d277427 ggml-cpu: gate __fp16 on __ARM_FP16_FORMAT_IEEE (llama/26860) 11e58f3e vulkan : dequant q8_0 KV once in coopmat1 (llama/25494) 4ef3e871 vulkan: add null checks in ggml_vk_queue_command_pools_cleanup (llama/27353) 689ad69f sycl: report zero devices instead of aborting when the host has none (llama/27291) 8442c74f ggml: add ggml_rope_set_offset (+ metal support) (llama/27120) d830bd22 metal : dequantize q8_0 using packed types (llama/27370) 1c882a8e vulkan: tiled transpose for 0<->2 permuted CONT (llama/26585) 7df5fa8e ggml-webgpu: add mulmat with overlapping src0/src1 (e.g., for minimax-01) (llama/27321) fa1e2bca opencl: fix WAR race in the generic FA tile kernels when the WG spans subgroups (llama/26434) 4be31016 RPC: populate use_count to enable fusion inside backends (llama/27142) dbab353f sycl: honor GGML_HINT_SRC0_IS_HADAMARD (llama/27298) c6a2bd09 devops : stop shadowing proper cuda libraries in runtime docker build (#3989) ec737355 tests: add check for nullptr for wctx in test-vad-full (#3998) a4ad15f3 ci : release clean-up (#4005) 81a3fadf server : only enable token timestamps when the response needs them (#3990) d61695dd docs : fix typos in readme files (#4004) b78df3d1 ci : move close-issue.yml to .github/workflows [no ci] (#4002) 339f2b4e bindings-javascript : remove package.json from git (#4001)