aqueduct LLM
Aqueduct allows you to run LLM and ML workloads on any infrastructure
...Aqueduct provides a single interface to running machine learning tasks on your existing cloud infrastructure — Kubernetes, Spark, Lambda, etc. From the same Python API, you can run code across any or all of these systems seamlessly and gain visibility into how your code is performing.