vLLM Integrations

Muse Spark Open is an open-weight, natively multimodal language model for developers and organizations building AI applications, coding systems, and agentic workflows. The released model has 816 billion total parameters, 42 billion active parameters, and a 1-million-token context window. Teams can download the model weights, run inference through supported engines such as vLLM, SGLang, TensorRT-LLM, Hugging Face Transformers, Apple MLX, llama.cpp, or Unsloth, and customize the model using PyTorch. It supports long-context reasoning, software engineering, coding, tool use, and multimodal inputs. Deployment options include cloud inference through listed API providers or self-hosting on compatible GPU systems; Meta documents a desktop setup using two Apple M5 Ultra Mac Studio computers. The model is released under the Muse Model License with limited commercial terms.

NVIDIA Nemotron 3.5 Bolt is a family of open artificial intelligence models designed for local AI agents, autonomous task execution, and conversational applications. The family includes Nemotron 3.5 Bolt 74B-A9B, a 74-billion-parameter mixture-of-experts model with approximately 9 billion active parameters, and Nemotron 3.5 Bolt E4B, a compact model optimized for conversational agents. The 74B-A9B variant uses native 2-bit quantization to operate within approximately 24GB of memory on compatible NVIDIA hardware. Its agentic capabilities support web research, software development, code debugging, and multistep task automation without requiring cloud-based inference. The E4B variant targets local voice assistants, interactive characters, and embedded applications with a 4GB memory footprint. Nemotron 3.5 Bolt supports deployment on NVIDIA RTX systems, DGX Spark, and other compatible hardware, giving developers greater control over inference, application data, and local AI workflows.

NVIDIA Nemotron 3.5 Super VL is an open, customizable multimodal AI model designed for complex reasoning, autonomous agents, and enterprise workflows. The model uses a 120-billion-parameter mixture-of-experts architecture that activates approximately 12 billion parameters per token to balance computational efficiency with model capabilities. It combines text, image, document, and video understanding with instruction following, coding, and external tool use. Its multimodal capabilities support document analysis, chart interpretation, optical character recognition, video understanding, and automated information extraction. Developers can customize the model through supervised fine-tuning, low-rank adaptation, and reinforcement learning using NVIDIA NeMo. Nemotron 3.5 Super VL supports deployment in local, cloud, and data center environments for applications such as software engineering, financial document processing, cybersecurity analysis, and enterprise automation.

Docker takes away repetitive, mundane configuration tasks and is used throughout the development lifecycle for fast, easy and portable application development, desktop and cloud. Docker’s comprehensive end-to-end platform includes UIs, CLIs, APIs and security that are engineered to work together across the entire application delivery lifecycle. Get a head start on your coding by leveraging Docker images to efficiently develop your own unique applications on Windows and Mac. Create your multi-container application using Docker Compose. Integrate with your favorite tools throughout your development pipeline, Docker works with all development tools you use including VS Code, CircleCI and GitHub. Package applications as portable container images to run in any environment consistently from on-premises Kubernetes to AWS ECS, Azure ACI, Google GKE and more. Leverage Docker Trusted Content, including Docker Official Images and images from Docker Verified Publishers.

Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery. Kubernetes builds upon 15 years of experience of running production workloads at Google, combined with best-of-breed ideas and practices from the community. Designed on the same principles that allows Google to run billions of containers a week, Kubernetes can scale without increasing your ops team. Whether testing locally or running a global enterprise, Kubernetes flexibility grows with you to deliver your applications consistently and easily no matter how complex your need is. Kubernetes is open source giving you the freedom to take advantage of on-premises, hybrid, or public cloud infrastructure, letting you effortlessly move workloads to where it matters to you.

OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity. We will attempt to directly build safe and beneficial AGI, but will also consider our mission fulfilled if our work aids others to achieve this outcome. Apply our API to any language task — semantic search, summarization, sentiment analysis, content generation, translation, and more — with only a few examples or by specifying your task in English. One simple integration gives you access to our constantly-improving AI technology. Explore how you integrate with the API with these sample completions.

DocsGPT is an open-source platform for private AI agents, assistants and enterprise search over an organisation's own documents, built by Arc53. It runs where the data is allowed to be: Arc53's EU cloud, a dedicated managed instance, the organisation's own servers, or fully air-gapped with no outbound traffic. Teams connect files and services (PDF, DOCX, XLSX, scanned documents with OCR, Google Drive, SharePoint, Confluence, GitHub, S3) and get answers cited to the source passage. Agents call tools and APIs with per-action approval, a visual builder turns document jobs into workflows, and agents can produce DOCX, XLSX, PPTX and PDF files. Any model works: OpenAI, Anthropic, Google and others in the cloud, or local models through Ollama or vLLM. Governance covers OIDC SSO, SCIM 2.0, role-based access, guardrails, usage quotas and an exportable, append-only audit log. MIT licence, 18,000+ GitHub stars.

Database Mart offers a comprehensive suite of server hosting solutions tailored for diverse computing needs. Their VPS hosting provides isolated CPU, memory, and disk resources with full root or admin access, supporting various applications such as database hosting, mail servers, file sharing, SEO tools, and script testing. These VPS plans come with SSD storage, automated backups, and an intuitive control panel, making them ideal for individuals and small businesses seeking cost-effective solutions. For more demanding applications, Database Mart's dedicated servers offer exclusive resources, ensuring superior performance and security. These servers are customizable to support large software systems and high-traffic e-commerce platforms, providing reliability for critical operations. Their GPU servers feature high-performance NVIDIA GPUs, catering to high-performance computing and advanced AI workloads.

omp is an open source AI coding agent and development harness that provides developers with a powerful local environment for AI-assisted engineering. It connects AI models directly to IDE capabilities, debugging tools, code execution, language servers, browser automation, memory, and dozens of built-in development tools. It supports more than 40 AI providers while allowing developers to use a single interface across cloud and local language models. omp enhances coding performance with features such as intelligent code editing, parallel subagents, persistent execution environments, integrated debugging, and advanced code review workflows. It also includes collaborative sessions, local memory, workflow automation, browser control, and GitHub integration to streamline complex software development tasks. Built with a native Rust engine and designed for Windows, macOS, and Linux, omp helps developers build, debug, and maintain software.

Highly scalable and standards-based model inference platform on Kubernetes for trusted AI. KServe is a standard model inference platform on Kubernetes, built for highly scalable use cases. Provides performant, standardized inference protocol across ML frameworks. Support modern serverless inference workload with autoscaling including a scale to zero on GPU. Provides high scalability, density packing, and intelligent routing using ModelMesh. Simple and pluggable production serving for production ML serving including prediction, pre/post-processing, monitoring, and explainability. Advanced deployments with the canary rollout, experiments, ensembles, and transformers. ModelMesh is designed for high-scale, high-density, and frequently-changing model use cases. ModelMesh intelligently loads and unloads AI models to and from memory to strike an intelligent trade-off between responsiveness to users and computational footprint.

Hugging Face is a leading platform for AI and machine learning, offering a vast hub for models, datasets, and tools for natural language processing (NLP) and beyond. The platform supports a wide range of applications, from text, image, and audio to 3D data analysis. Hugging Face fosters collaboration among researchers, developers, and companies by providing open-source tools like Transformers, Diffusers, and Tokenizers. It enables users to build, share, and access pre-trained models, accelerating AI development for a variety of industries.

Thunder Compute is a GPU cloud for developers. It features competitive pricing, simple UX, and pre-built tools for common AI workflows. Most neoclouds are real-estate companies; they build data centers, with software as an afterthought. That software is what developers see, touch, and interact with every day. It is critical. This team of cracked systems and infrastructure engineers is flipping that script. They're building the most enjoyable, low-cost, reliable GPU cloud for developers.

NGINX Open Source: The open source web server that powers more than 400 million websites. NGINX Plus is a software load balancer, web server, and content cache built on top of open source NGINX. Use NGINX Plus instead of your hardware load balancer and get the freedom to innovate without being constrained by infrastructure. Save more than 80% compared to hardware ADCs, without sacrificing performance or functionality. Deploy anywhere: public cloud, private cloud, bare metal, virtual machines, and containers. Save time by performing common tasks through the built‑in NGINX Plus API. From NetOps to DevOps, modern app teams need a self‑service, API‑driven platform that integrates easily into CI/CD workflows to accelerate app deployment – whether your app has a hybrid or microservices architecture – and makes app lifecycle management easier.

Transition seamlessly between eager and graph modes with TorchScript, and accelerate the path to production with TorchServe. Scalable distributed training and performance optimization in research and production is enabled by the torch-distributed backend. A rich ecosystem of tools and libraries extends PyTorch and supports development in computer vision, NLP and more. PyTorch is well supported on major cloud platforms, providing frictionless development and easy scaling. Select your preferences and run the install command. Stable represents the most currently tested and supported version of PyTorch. This should be suitable for many users. Preview is available if you want the latest, not fully tested and supported, 1.10 builds that are generated nightly. Please ensure that you have met the prerequisites (e.g., numpy), depending on your package manager. Anaconda is our recommended package manager since it installs all dependencies.

Software is what turns a vehicle into an intelligent machine. The NVIDIA DRIVE™ Software stack is open, empowering developers to efficiently build and deploy a variety of state-of-the-art AV applications, including perception, localization and mapping, planning and control, driver monitoring, and natural language processing. The foundation of the DRIVE Software stack, DRIVE OS is the first safe operating system for accelerated computing. It includes NvMedia for sensor input processing, NVIDIA CUDA® libraries for efficient parallel computing implementations, NVIDIA TensorRT™ for real-time AI inference, and other developer tools and modules to access hardware engines. The NVIDIA DriveWorks® SDK provides middleware functions on top of DRIVE OS that are fundamental to autonomous vehicle development. These consist of the sensor abstraction layer (SAL) and sensor plugins, data recorder, vehicle I/O support, and a deep neural network (DNN) framework.