Search Results for "grpc"

Jina Serve is an open-source framework designed for building, deploying, and scaling AI services and machine learning pipelines in production environments. The framework allows developers to create microservices that expose machine learning models through APIs that communicate using protocols such as HTTP, gRPC, and WebSockets. It is built with a cloud-native architecture that supports deployment on local machines, containerized environments, or large orchestration platforms such as Kubernetes. Jina Serve focuses on making it easier to turn machine learning models into production-ready services without forcing developers to manage complex infrastructure manually. ...

...Build applications that deliver fresh insights from multiple data types such as text, image, audio, video, 3D mesh, PDF with Jina AI’s DocArray. Polyglot gateway that supports gRPC, Websockets, HTTP, GraphQL protocols with TLS. Intuitive design pattern for high-performance microservices. Seamless Docker container integration: sharing, exploring, sandboxing, versioning and dependency control via Jina Hub. Fast deployment to Kubernetes, Docker Compose and Jina Cloud. Improved engineering efficiency thanks to the Jina AI ecosystem, so you can focus on innovating with the data applications you build.

...Provides Backend API that allows adding custom backends and pre/post-processing operations. Model pipelines using Ensembling or Business Logic Scripting (BLS). HTTP/REST and GRPC inference protocols based on the community-developed KServe protocol. A C API and Java API allow Triton to link directly into your application for edge and other in-process use cases.

BentoML simplifies ML model deployment and serves your models at a production scale. Support multiple ML frameworks natively: Tensorflow, PyTorch, XGBoost, Scikit-Learn and many more! Define custom serving pipeline with pre-processing, post-processing and ensemble models. Standard .bento format for packaging code, models and dependencies for easy versioning and deployment. Integrate with any training pipeline or ML experimentation platform. Parallelize compute-intense model inference...

...No learning curve, minimalist design on client and server. Intuitive and consistent API for image and sentence embedding. Async client support. Easily switch between gRPC, HTTP, WebSocket protocols with TLS and compression. Smooth integration with neural search ecosystem including Jina and DocArray. Build cross-modal and multi-modal solutions in no time.