Search Results for "frameworks"

Mooncake is an open-source infrastructure platform designed to optimize large language model serving by focusing on efficient management and transfer of model data and KV cache. The platform was originally developed as part of the serving infrastructure for the Kimi large language model system. Its architecture centers on a high-performance transfer engine that provides unified data transfer across different storage and networking technologies. This engine enables efficient movement of...

...The project is designed for developers who need direct access to the company’s Tensix processor architecture, exposing a programming model that is closer to hardware control than high-level inference frameworks. Instead of following a traditional GPU model centered on massive thread parallelism, the platform is built around a grid of specialized compute nodes called Tensix cores, each with local SRAM, dedicated compute units, and multiple RISC-V control processors. The SDK provides the abstractions and APIs needed to manage data movement, compute kernels, memory coordination, and execution flow across this architecture.

nndeploy is an open-source framework designed to simplify the deployment of artificial intelligence models across multiple hardware platforms and devices. The framework focuses on making it easier to transform trained AI models into production-ready applications that can run efficiently on desktops, mobile devices, servers, and edge computing hardware. Developers can use visual workflows to design and configure AI processing pipelines by connecting modular nodes that represent different...