Sangfor aStorSangfor
|
||||||
Related Products
|
||||||
About
LMCache is an open source Knowledge Delivery Network (KDN) designed as a caching layer for large language model serving that accelerates inference by reusing KV (key-value) caches across repeated or overlapping computations. It enables fast prompt caching, allowing LLMs to “prefill” recurring text only once and then reuse those stored KV caches, even in non-prefix positions, across multiple serving instances. This approach reduces time to first token, saves GPU cycles, and increases throughput in scenarios such as multi-round question answering or retrieval augmented generation. LMCache supports KV cache offloading (moving cache from GPU to CPU or disk), cache sharing across instances, and disaggregated prefill, which separates the prefill and decoding phases for resource efficiency. It is compatible with inference engines like vLLM and TGI and supports compressed storage, blending techniques to merge caches, and multiple backend storage options.
|
About
Sangfor aStor is a software‑defined storage solution that unifies block, file, and object storage into a single, elastically expandable resource pool using a fully symmetrical distributed architecture, enabling on‑demand allocation of high‑performance and cost‑optimized, large‑capacity tiers to suit diverse service requirements. Available as either integrated hardware‑software or standalone software, it scales from just three commodity x86 nodes and supports cloud‑scale clusters of thousands of nodes with EB‑level capacity expansion. Its multi‑node parallel processing and intelligent caching (using RDMA, SSD hot‑data cache, and layering) deliver extremely high throughput, IOPS, and small‑IO performance, boosting cache hit rates to 90% and small‑IO handling by up to 65%, while distributed metadata management ensures jitter‑free handling of billions of files.
|
|||||
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
|||||
Audience
AI engineers and infrastructure teams looking for a tool to lower latency, reduce compute cost, and scale throughput
|
Audience
Enterprises wanting a solution to manage block, file, and object workloads with minimal operational overhead
|
|||||
Support
Phone Support
24/7 Live Support
Online
|
Support
Phone Support
24/7 Live Support
Online
|
|||||
API
Offers API
|
API
Offers API
|
|||||
Screenshots and Videos |
Screenshots and Videos |
|||||
Pricing
Free
Free Version
Free Trial
|
Pricing
No information available.
Free Version
Free Trial
|
|||||
Reviews/
|
Reviews/
|
|||||
Training
Documentation
Webinars
Live Online
In Person
|
Training
Documentation
Webinars
Live Online
In Person
|
|||||
Company InformationLMCache
United States
lmcache.ai/
|
Company InformationSangfor
Founded: 2000
China
www.sangfor.com/cloud-and-infrastructure/products/astor-enterprise-data-storage-solution
|
|||||
Alternatives |
Alternatives |
|||||
|
|
||||||
|
|
||||||
|
|
|
|||||
|
|
||||||
Categories |
Categories |
|||||
Integrations
Amazon S3
Microsoft Hyper-V
OpenStack
Swift
VMware Cloud
|
||||||
|
|
|