Jlama
Jlama is a modern LLM inference engine for Java
Jlama is a modern inference engine written entirely in Java that enables developers to run large language models locally within Java applications. Unlike frameworks that require external APIs or remote services, Jlama performs inference directly on a machine using pre-trained models. This allows organizations to integrate generative AI features into their systems while maintaining full control over data privacy and infrastructure. The engine supports a wide range of open-source model architectures and formats, including variants of Llama, Mistral, and other transformer-based models. ...