Related Products
|
||||||
About
LMCache is an open source Knowledge Delivery Network (KDN) designed as a caching layer for large language model serving that accelerates inference by reusing KV (key-value) caches across repeated or overlapping computations. It enables fast prompt caching, allowing LLMs to “prefill” recurring text only once and then reuse those stored KV caches, even in non-prefix positions, across multiple serving instances. This approach reduces time to first token, saves GPU cycles, and increases throughput in scenarios such as multi-round question answering or retrieval augmented generation. LMCache supports KV cache offloading (moving cache from GPU to CPU or disk), cache sharing across instances, and disaggregated prefill, which separates the prefill and decoding phases for resource efficiency. It is compatible with inference engines like vLLM and TGI and supports compressed storage, blending techniques to merge caches, and multiple backend storage options.
|
About
PrimaryIO’s HDM technology decouples compute and storage to seamlessly move workloads to and from the cloud rapidly and lets you fully control data. HDM provides workload mobility and data control with a combination of innovative technologies, cloud cache, cloud storage gateway, and an integrated smart IO analyzer. A smart IO analyzer transparently monitors virtual machine-to-datastore access patterns to identify frequently accessed data (hot set) and provide recommendations. HDM storage gateways move only the hot data to the cloud and maintain secure connectivity to the persistent on-premises data store. HDM’s dynamic and secure cloud cache helps to rapidly spin up instances in the cloud. HDM integrates into VMware vCenter as a plug-in and all operations are transparently handled without impacting applications. Rapid cloud testing for risk-free migrations. Agility to rapidly lift and shift workloads or roll back at will. Leverage the cloud for the seasonal demand.
|
|||||
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
Platforms Supported
Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook
|
|||||
Audience
AI engineers and infrastructure teams looking for a tool to lower latency, reduce compute cost, and scale throughput
|
Audience
Enterprises in need of a solution to move workloads to and from the cloud while controlling their data
|
|||||
Support
Phone Support
24/7 Live Support
Online
|
Support
Phone Support
24/7 Live Support
Online
|
|||||
API
Offers API
|
API
Offers API
|
|||||
Screenshots and Videos |
Screenshots and Videos |
|||||
Pricing
Free
Free Version
Free Trial
|
Pricing
No information available.
Free Version
Free Trial
|
|||||
Reviews/
|
Reviews/
|
|||||
Training
Documentation
Webinars
Live Online
In Person
|
Training
Documentation
Webinars
Live Online
In Person
|
|||||
Company InformationLMCache
United States
lmcache.ai/
|
Company InformationPrimaryIO
Founded: 2012
United States
www.primaryio.com/hybrid-cloud-data-mgmt/
|
|||||
Alternatives |
Alternatives |
|||||
|
|
|
|||||
|
|
|
|||||
|
|
||||||
Categories |
Categories |
|||||
Integrations
VMware vCenter
|
||||||
|
|
|