Open Source Java Large Language Models (LLM)

Confidently evaluate, test, and monitor LLM applications. Opik is an open-source platform for evaluating, testing, and monitoring LLM applications. Built by Comet. Record, sort, search, and understand each step your LLM app takes to generate a response. Manually annotate, view, and compare LLM responses in a user-friendly table. Log traces during development and in production. Run experiments with different prompts and evaluate against a test set. Choose and run pre-configured evaluation metrics or define your own with our convenient SDK library. Consult built-in LLM judges for complex issues like hallucination detection, factuality, and moderation.

LangChain for Java is an open-source Java framework designed to simplify the development of applications powered by large language models. The library provides a unified API that allows developers to connect Java applications to multiple AI providers and embedding databases without having to implement separate integrations for each service. Its architecture includes abstractions for prompts, chat interactions, document processing, embeddings, and vector storage, enabling developers to build complex AI workflows with minimal boilerplate code. LangChain4j also implements common design patterns used in generative AI systems, such as retrieval-augmented generation pipelines, tool calling, and intelligent agent frameworks. These abstractions allow developers to orchestrate interactions between language models, external tools, and knowledge bases in a structured and scalable way.

Google’s Agent Development Kit for Java is an open-source toolkit that helps developers design, evaluate, and deploy advanced AI agents using the Java programming language. The framework follows a code-first approach that treats agent development as a structured software engineering task rather than a collection of prompt scripts. It provides abstractions and tools that allow developers to create agents capable of executing complex workflows, calling tools, and interacting with external services. ADK is designed to be flexible and modular so that developers can build simple automation agents or large distributed agent systems depending on their needs. While it integrates well with Google’s AI ecosystem, the framework is designed to remain model-agnostic and compatible with different machine learning platforms.

JADX-AI-MCP is an open-source plugin that integrates large language models into the JADX Android decompiler to assist with reverse engineering and code analysis tasks. The project connects JADX with AI assistants through the Model Context Protocol, enabling language models to interact directly with decompiled Android application code. Through this integration, AI systems can inspect classes, analyze methods, retrieve application manifests, and examine other elements of Android packages in real time. The plugin works alongside a companion MCP server that exposes reverse engineering tools to AI clients so they can query and analyze code programmatically. This allows developers and security researchers to perform contextual code reviews and vulnerability analysis using AI-assisted workflows.

Jlama is a modern inference engine written entirely in Java that enables developers to run large language models locally within Java applications. Unlike frameworks that require external APIs or remote services, Jlama performs inference directly on a machine using pre-trained models. This allows organizations to integrate generative AI features into their systems while maintaining full control over data privacy and infrastructure. The engine supports a wide range of open-source model architectures and formats, including variants of Llama, Mistral, and other transformer-based models. It provides tools for running chat interactions, completing prompts, or exposing an OpenAI-compatible REST API for applications that expect standard LLM endpoints. The project focuses on performance and portability by using native Java optimizations and the Java Vector API to accelerate inference workloads.