ZML

ZML is a high-performance machine learning inference stack designed to run AI models efficiently across heterogeneous hardware environments using a modern systems programming approach. Built with technologies such as Zig, MLIR, and Bazel, it focuses on production-grade deployment where performance, portability, and scalability are critical. The system allows models to be compiled and executed across multiple types of accelerators, including GPUs and TPUs, even when distributed across different machines or locations. One of its key strengths is cross-compilation, enabling developers to build once and deploy across various platforms without rewriting code. zml provides example implementations of models and workflows, demonstrating how to run inference tasks such as image classification or large language models. It is designed to handle complex distributed setups, including scenarios where model components are split across devices connected via networks.