KServe

NVIDIA Triton™ inference server delivers fast and scalable AI in production. Open-source inference serving software, Triton inference server streamlines AI inference by enabling teams deploy trained AI models from any framework (TensorFlow, NVIDIA TensorRT®, PyTorch, ONNX, XGBoost, Python, custom and more on any GPU- or CPU-based infrastructure (cloud, data center, or edge). Triton runs models concurrently on GPUs to maximize throughput and utilization, supports x86 and ARM CPU-based inferencing, and offers features like dynamic batching, model analyzer, model ensemble, and audio streaming. Triton helps developers deliver high-performance inference aTriton integrates with Kubernetes for orchestration and scaling, exports Prometheus metrics for monitoring, supports live model updates, and can be used in all major public cloud machine learning (ML) and managed Kubernetes platforms. Triton helps standardize model deployment in production.

Our industry-leading single socket or dual socket high-performance CIARA ORION High Density (HD) servers offer unmatched flexibility, scalability, and efficiency to handle all your critical workloads. The ORION HD products offer the industry’s best density of cores per rackmount unit in order to guarantee optimal rackmount utilization in any data center. Compatible with both Intel® Xeon® Processor Scalable Family and AMD EPYC® processors, ORION High-Density servers provide incredible design options for large-scale deployment of high-density IT and HPC workloads. The ORION high-density server product line is built with the latest silicon technology to provide the best performance, and support of the highest TDP of the industry alongside a vast array of storage options and extensive add-on card support. It is ideal for infrastructure consolidation, academic research, cloud & hosting providers as well as high-performance computing applications.