Mooncake Files

What's Changed

• [Store]wrapper etcd basic interface for master service HA by @Libotry in https://github.com/kvcache-ai/Mooncake/pull/1451
• [Build] Update package name and keywords for CUDA 13 build by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1506
• [EP] Improve debug message by @UNIDY2002 in https://github.com/kvcache-ai/Mooncake/pull/1505
• [Misc] fix wheel build script by @Ann-1024 in https://github.com/kvcache-ai/Mooncake/pull/1504
• [Store][Feature] copy and move client support by @zhongzhouTan-coder in https://github.com/kvcache-ai/Mooncake/pull/1364
• [EP] fix a regression when IBGDA is disabled by @UNIDY2002 in https://github.com/kvcache-ai/Mooncake/pull/1514
• Add ROLL collaboration announcement to README Updates section by @Copilot in https://github.com/kvcache-ai/Mooncake/pull/1517
• [TE] Add AWS EFA transport using libfabric by @whn09 in https://github.com/kvcache-ai/Mooncake/pull/1509
• [TENT] fix: Reduce unnecessary bandwidth consumption in TCP recvData operations by @00fish0 in https://github.com/kvcache-ai/Mooncake/pull/1513
• [Config][1/n] add global config for all env variables by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1512
• [docs] add conductor indexer api design doc by @yejj710 in https://github.com/kvcache-ai/Mooncake/pull/1416
• [Doc] Document missing USE_MNNVL compile option by @ShangmingCai in https://github.com/kvcache-ai/Mooncake/pull/1525
• [TE] feat: hixl support report errmsg when interface called failed by @A-Liuhao in https://github.com/kvcache-ai/Mooncake/pull/1524
• [TE] Support TCP fallback in EFA build and improve EFA documentation by @whn09 in https://github.com/kvcache-ai/Mooncake/pull/1523
• [Store] - Optimize BucketStorageBackend for reduced lock contention and add delete safety by @maheshrbapatu in https://github.com/kvcache-ai/Mooncake/pull/1456
• Add Local Cache Mechanism for Mooncake Store Client by @Shichang-Zhang in https://github.com/kvcache-ai/Mooncake/pull/1226
• [CI]Add sglang epd testcases by @luketong777 in https://github.com/kvcache-ai/Mooncake/pull/1528
• add efa protocol to mooncake store client by @snadampal in https://github.com/kvcache-ai/Mooncake/pull/1526
• implement cpp methods for p2p connection by @donghun-furiosa in https://github.com/kvcache-ai/Mooncake/pull/1539
• [TENT] Improve tebench: GPU selection, graceful interruption, and build fixes by @staryxchen in https://github.com/kvcache-ai/Mooncake/pull/1537
• [TE] change ascend direct transport docs & fix async transfer disconnect bug by @ascend-direct-dev in https://github.com/kvcache-ai/Mooncake/pull/1534
• [Bug]add ci switch and free space codes by @JasonZhang517 in https://github.com/kvcache-ai/Mooncake/pull/1540
• [Store] Introduce Free-Ratio-First Allocation Strategy to Improve Convergence by @00fish0 in https://github.com/kvcache-ai/Mooncake/pull/1511
• Update README with recent project updates by @zhyncs in https://github.com/kvcache-ai/Mooncake/pull/1541
• [TE] update logging for memory type in AscendDirectTransport by @Cheng-China in https://github.com/kvcache-ai/Mooncake/pull/1542
• [TE] add ascend direct transport unit test by @ascend-direct-dev in https://github.com/kvcache-ai/Mooncake/pull/1543
• [TE] Ubshmem transport support ipc memory and build allocator when set USE_UBSHMEM=ON by @VNightMare in https://github.com/kvcache-ai/Mooncake/pull/1519
• [Doc] Add Mooncake Python API skill for Claude Code by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1545
• [Store]add unit test for exception handling of RealClient by @dongb0 in https://github.com/kvcache-ai/Mooncake/pull/1546
• [TE/HIP] Support ROCm 7.1+ hipMemImportFromShareableHandle signature change by @amd-arozanov in https://github.com/kvcache-ai/Mooncake/pull/1550
• [Store] feat: CXL storage full features. by @qiuweit7 in https://github.com/kvcache-ai/Mooncake/pull/1531
• [PG] Implement efficient P2P proxy for low-latency send/recv communication by @yuechen-sys in https://github.com/kvcache-ai/Mooncake/pull/1533
• [DOC] Update Slack link in README.md by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1552
• [Store]Implement a file interface based on io uring to optimize storage backend. by @zhangzuo21 in https://github.com/kvcache-ai/Mooncake/pull/1500
• [Store] Enable dummy client zero-copy get_buffer via shared hot cache by @YiXR in https://github.com/kvcache-ai/Mooncake/pull/1535
• [Misc] add new dockerfile by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1214
• [CI] Fix CI test cases in sglang containers by @luketong777 in https://github.com/kvcache-ai/Mooncake/pull/1555
• [CI] chore: update codeowner by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1556
• [Store] add safe tensor API by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1199
• [CI]Add sglang elastic ep backend testcases by @hhr2449 in https://github.com/kvcache-ai/Mooncake/pull/1561
• add ieisystem logo to contributors by @qiuweit7 in https://github.com/kvcache-ai/Mooncake/pull/1557
• [PG] Fix several memory leaks in MooncakeBackend by @caozhanhao in https://github.com/kvcache-ai/Mooncake/pull/1551
• [CI]extend waiting time for aritifact by @Ann-1024 in https://github.com/kvcache-ai/Mooncake/pull/1563
• [TENT] fix(RDMA): remove bootstrap RPC re-entrancy deadlock via self-contained handshake by @00fish0 in https://github.com/kvcache-ai/Mooncake/pull/1560
• [TE] Implement TCP connection pooling to reduce connection overhead by @alogfans in https://github.com/kvcache-ai/Mooncake/pull/1508
• [TENT] feat: Speed up startup memory registration: NUMA prefault + RDMA MR warm‑up by @00fish0 in https://github.com/kvcache-ai/Mooncake/pull/1468
• [DOCS] update pull request template by @staryxchen in https://github.com/kvcache-ai/Mooncake/pull/1568
• [PG/EP]: fix EP/PG extension linking and avoid patchelf CUDA fatbin corruption by @Socratesa in https://github.com/kvcache-ai/Mooncake/pull/1571
• docs: add TorchSpec in Mooncake README by @zhyncs in https://github.com/kvcache-ai/Mooncake/pull/1574
• [Bugfix] Fix RDMA notification send buffer DMA race and reconnect hang by @staryxchen in https://github.com/kvcache-ai/Mooncake/pull/1567
• [Doc] update README by @zhyncs in https://github.com/kvcache-ai/Mooncake/pull/1575
• [CI] feat(tests): support vLLM version-based proxy selection and unbuffered logging by @luketong777 in https://github.com/kvcache-ai/Mooncake/pull/1565
• refactor(rdma): replace C-style arrays with std::vector for work requests by @staryxchen in https://github.com/kvcache-ai/Mooncake/pull/1576
• [TE] [STORE] Improve UBShmem Transport performance with stream pool && adapts to Mooncake Store by @VNightMare in https://github.com/kvcache-ai/Mooncake/pull/1591
• [TENT] Fix resource cleanup order to prevent SubBatch/Slice leak by @00fish0 in https://github.com/kvcache-ai/Mooncake/pull/1585
• p2p-store: expose GetLocalIpAndPort/GetLocalServerName in Go API by @lclgo in https://github.com/kvcache-ai/Mooncake/pull/1579
• [Doc]: add allocation strategy documentation with user guidance by @00fish0 in https://github.com/kvcache-ai/Mooncake/pull/1577
• [Store] Implement Metadata Persistence and Recovery for Master Service by @yangdao479 in https://github.com/kvcache-ai/Mooncake/pull/1431
• [DOC] Revise pip install commands for CUDA versions by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1595
• [CI] add CU13_BUILD environment variable to CI workflow by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1590
• [TE] Fix remaining race conditions in removeSegmentDesc and updateLocalSegmentDesc by @DukeDeSouth in https://github.com/kvcache-ai/Mooncake/pull/1599
• [Bug fix]fix compile err without use uring by @zhangzuo21 in https://github.com/kvcache-ai/Mooncake/pull/1602
• [Store] Notify master on disk eviction to fix stale metadata by @duhaode520 in https://github.com/kvcache-ai/Mooncake/pull/1549
• [refactor] move common ResolvePath to utils by @chunxiaozheng in https://github.com/kvcache-ai/Mooncake/pull/1610
• fix: check MC_INTRANODE_NVLINK before HCA auto-detection fallback by @ishandhanani in https://github.com/kvcache-ai/Mooncake/pull/1612
• [Store] Remove duplicate code in allocate/free ascend fabric memory function of mooncake store by @VNightMare in https://github.com/kvcache-ai/Mooncake/pull/1598
• [Build] fix missing libcurl4 dependency in mooncake docker image by @TrafalgarZZZ in https://github.com/kvcache-ai/Mooncake/pull/1619
• Remove py3.8 logic from scripts/build_wheel.sh by @Copilot in https://github.com/kvcache-ai/Mooncake/pull/1615
• [Store] Add health check API for Client with HTTP /health endpoint by @duhaode520 in https://github.com/kvcache-ai/Mooncake/pull/1606
• [PG] Share P2PProxy/ConnectionPoller threads across backends. by @caozhanhao in https://github.com/kvcache-ai/Mooncake/pull/1607
• [EP] In-place Member Update by @ympcMark in https://github.com/kvcache-ai/Mooncake/pull/1630
• [PG] Remove CPU-only backend tests from CI by @UNIDY2002 in https://github.com/kvcache-ai/Mooncake/pull/1628
• Fix EP buffer allocation for MNNVL clusters by @he-yufeng in https://github.com/kvcache-ai/Mooncake/pull/1629
• [TE] remove target segment desc cache when disconnect by @ascend-direct-dev in https://github.com/kvcache-ai/Mooncake/pull/1624
• [TransferEngine] Fix RDMA GID auto-discovery for IPv6 and reduce spur… by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1597
• [Store] [TE] Refactor mem allocation process in ascend platform by @ascend-direct-dev in https://github.com/kvcache-ai/Mooncake/pull/1623
• [DOC] update news and badges readme by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1636
• [Store] Add /metrics and /metrics/summary HTTP endpoints to RealClient by @duhaode520 in https://github.com/kvcache-ai/Mooncake/pull/1634
• [CI]skip integration test for non-core file changes by @luketong777 in https://github.com/kvcache-ai/Mooncake/pull/1609
• [EP] Enable Fabric Mem only if MC_USE_NVLINK_IPC is explicitly set to zero by @UNIDY2002 in https://github.com/kvcache-ai/Mooncake/pull/1637
• [Store] Fix error log spam for non-memory replicas in DiscardedReplicas by @he-yufeng in https://github.com/kvcache-ai/Mooncake/pull/1626
• [Store] bugfix: fix signal missing by @wanyue-wy in https://github.com/kvcache-ai/Mooncake/pull/1632
• fix qp_leak bug after PR#384 by @litiantian00 in https://github.com/kvcache-ai/Mooncake/pull/1643
• [Store]Implement allocation strategy benchmark by @dongb0 in https://github.com/kvcache-ai/Mooncake/pull/1587
• [CI] Add CI workflow on ASCEND platform by @VNightMare in https://github.com/kvcache-ai/Mooncake/pull/1640
• [Bugfix][HA] Disable client_pool alive_detect to stop stale reconnection logs after HA master failover by @duhaode520 in https://github.com/kvcache-ai/Mooncake/pull/1642
• [Store]Optimize uring file support for SSD offloading by @zhangzuo21 in https://github.com/kvcache-ai/Mooncake/pull/1562
• Fix NVLink IPC offset corruption for sub-allocated GPU tensors by @ishandhanani in https://github.com/kvcache-ai/Mooncake/pull/1622
• [Store] NUMA-Aware Global Segment Allocation for Full RDMA NIC Utilization in Standalone Mode by @YiXR in https://github.com/kvcache-ai/Mooncake/pull/1580
• Move EP/PG wheel-building logic from build_wheel.sh into CMake by @Copilot in https://github.com/kvcache-ai/Mooncake/pull/1616
• [TE] add retry logic for ascend direct by @ascend-direct-dev in https://github.com/kvcache-ai/Mooncake/pull/1641
• [Store] Fix Ctrl-C hang in both Python and C++ client processes by @00fish0 in https://github.com/kvcache-ai/Mooncake/pull/1620
• [Store] put tensor zero copy by @zxpdemonio in https://github.com/kvcache-ai/Mooncake/pull/1480
• Fix MNNVL warmup hang: skip warmup when fabric mem is available by @he-yufeng in https://github.com/kvcache-ai/Mooncake/pull/1644
• [PG] force register local memory on specific devices by @UNIDY2002 in https://github.com/kvcache-ai/Mooncake/pull/1656
• [PG] fix: handle null backend options in Mooncake PG init by @yuechen-sys in https://github.com/kvcache-ai/Mooncake/pull/1649
• [STORE] feat: Frequency admission + LRU lock optimization for local hot cache by @chenwenxiaolive in https://github.com/kvcache-ai/Mooncake/pull/1596
• fix compile error when use -DUSE_ASCEND_HETEROGENEOUS=ON by @xleoken in https://github.com/kvcache-ai/Mooncake/pull/1663
• [CI] fix musa build hang issue by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1666
• [CI] force reinstall mooncake wheel by @luketong777 in https://github.com/kvcache-ai/Mooncake/pull/1667
• [CI] avoid frequent website deployment by @stmatengss in https://github.com/kvcache-ai/Mooncake/pull/1671
• SSD benchmarks based on Mooncake Trace by @alogfans in https://github.com/kvcache-ai/Mooncake/pull/1613
• [Store][Feature]Hot Standby and Oplog Interface for Master Service HA by @Libotry in https://github.com/kvcache-ai/Mooncake/pull/1515
• [CI] update ascend ci workflow and docs by @VNightMare in https://github.com/kvcache-ai/Mooncake/pull/1683
• Modify CODEOWNERS to add new code owners by @UNIDY2002 in https://github.com/kvcache-ai/Mooncake/pull/1679
• fix: fix heterogeneous rdma transport error by @XingSong-Sun in https://github.com/kvcache-ai/Mooncake/pull/1657
• [PG] force register local memory for P2P memory regions by @UNIDY2002 in https://github.com/kvcache-ai/Mooncake/pull/1690
• [PG]: Initialize Mooncake PG integration for TENT by @KMSorSMS in https://github.com/kvcache-ai/Mooncake/pull/1676
• Bump version to 0.3.10 in pyproject.toml by @ShangmingCai in https://github.com/kvcache-ai/Mooncake/pull/1693
• Fix client_ttl flag name check in master by @Primary33 in https://github.com/kvcache-ai/Mooncake/pull/1691
• Enhance build_wheel.sh to support uv as well by @ShangmingCai in https://github.com/kvcache-ai/Mooncake/pull/1694
• [TE] fallback to 2M malloc by @ascend-direct-dev in https://github.com/kvcache-ai/Mooncake/pull/1687
• [Docs] Add SSD offload documentation by @zhangzuo21 in https://github.com/kvcache-ai/Mooncake/pull/1647
• [Store]Fix unnecessary value copies in mooncake-store by @SYaoJun in https://github.com/kvcache-ai/Mooncake/pull/1672
• [TE] Remove false-positive slice leak detection in ThreadLocalSliceCache by @00fish0 in https://github.com/kvcache-ai/Mooncake/pull/1703
• [CI] Update artifact selection criteria in workflow by @UNIDY2002 in https://github.com/kvcache-ai/Mooncake/pull/1700

New Contributors

• @Libotry made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1451
• @whn09 made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1509
• @A-Liuhao made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1524
• @Shichang-Zhang made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1226
• @snadampal made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1526
• @donghun-furiosa made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1539
• @dongb0 made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1546
• @Socratesa made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1571
• @DukeDeSouth made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1599
• @duhaode520 made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1549
• @chunxiaozheng made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1610
• @ishandhanani made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1612
• @TrafalgarZZZ made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1619
• @he-yufeng made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1629
• @litiantian00 made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1643
• @xleoken made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1663
• @XingSong-Sun made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1657
• @Primary33 made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1691
• @SYaoJun made their first contribution in https://github.com/kvcache-ai/Mooncake/pull/1672

Full Changelog: https://github.com/kvcache-ai/Mooncake/compare/v0.3.9...v0.3.10