A high-throughput and memory-efficient inference and serving engine
Run Local LLMs on Any Device. Open-source
Ready-to-use OCR with 80+ supported languages
The official Python client for the Huggingface Hub
A unified framework for scalable computing
Neural Network Compression Framework for enhanced OpenVINO
Tensor search for humans
Library for OCR-related tasks powered by Deep Learning
Phi-3.5 for Mac: Locally-run Vision and Language Models
Single-cell analysis in Python
Operating LLMs in production
Large Language Model Text Generation Inference
Libraries for applying sparsification recipes to neural networks
An easy-to-use LLMs quantization package with user-friendly apis
Gaussian processes in TensorFlow
Everything you need to build state-of-the-art foundation models
A high-performance ML model serving framework, offers dynamic batching
Openai style api for open large language models
Sparsity-aware deep learning inference runtime for CPUs
Uncover insights, surface problems, monitor, and fine tune your LLM
FlashInfer: Kernel Library for LLM Serving
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
State-of-the-art diffusion models for image and audio generation
Efficient few-shot learning with Sentence Transformers
Trainable models and NN optimization tools