Search Results for "linux deploy"
Sort By:
Showing 23 open source projects for "linux deploy"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
OpenLLM
Operating LLMs in production
An open platform for operating large language models (LLMs) in production. Fine-tune, serve, deploy, and monitor any LLMs with ease. With OpenLLM, you can run inference with any open-source large-language models, deploy to the cloud or on-premises, and build powerful AI apps. Built-in supports a wide range of open-source LLMs and model runtime, including Llama 2， StableLM, Falcon, Dolly, Flan-T5, ChatGLM, StarCoder, and more. Serve LLMs over RESTful API or gRPC with one command, query via... -
2
LlamaDeploy
Deploy your agentic worfklows to production
llama_deploy is an open-source framework designed to simplify the deployment and productionization of agent-based AI workflows built with the LlamaIndex ecosystem. The project provides an asynchronous architecture that allows developers to deploy complex multi-agent workflows as scalable microservices. It enables teams to move from experimental prototypes to production systems with minimal changes to existing LlamaIndex code, making it easier to operationalize AI agents. The system supports... -
3
Xorbits Inference
Replace OpenAI GPT with another LLM in your app
Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop. Xorbits Inference(Xinference) is a powerful and versatile library designed to serve language, speech recognition, and multimodal models. With Xorbits... -
4
Infinity
Low-latency REST API for serving text-embeddings
Infinity is a high-throughput, low-latency REST API for serving vector embeddings, supporting all sentence-transformer models and frameworks. Infinity is developed under MIT License. Infinity powers inference behind Gradient.ai and other Embedding API providers. -
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
5
Guidance
A guidance language for controlling large language models
Guidance is an efficient programming paradigm for steering language models. With Guidance, you can control how output is structured and get high-quality output for your use case—while reducing latency and cost vs. conventional prompting or fine-tuning. It allows users to constrain generation (e.g. with regex and CFGs) as well as to interleave control (conditionals, loops, tool use) and generation seamlessly. -
6
Cybersecurity AI
Cybersecurity AI (CAI), the framework for AI Security
CAI (Cybersecurity AI) is a lightweight open-source framework intended to help security practitioners build and deploy AI-assisted automation for defensive and offensive security workflows. The project frames itself as a practical foundation for “AI security,” focusing on turning security tasks into agentic workflows that can be composed, executed, and iterated on by practitioners. Rather than being a single-purpose tool, CAI is positioned as a framework that supports building multiple... -
7
GPUStack
Performance-optimized AI inference on your GPUs
...Instead of requiring complex orchestration systems such as Kubernetes, GPUStack provides a lightweight environment that automatically selects appropriate inference engines, configures deployment parameters, and schedules workloads across available GPUs. The platform supports GPUs from a wide range of vendors and can run on laptops, workstations, and servers across operating systems such as macOS, Windows, and Linux. It also enables developers to deploy models from common repositories like Hugging Face and access them through APIs similar to cloud-based AI services. -
8
LangBot
Production-grade platform for building agentic IM bots
LangBot is an open source platform designed to build and deploy AI-powered chatbots across multiple instant messaging ecosystems. The system allows developers to integrate large language models into messaging platforms so that bots can perform tasks, answer questions, and automate workflows directly within everyday communication tools. It supports numerous messaging services including Discord, Slack, Telegram, WeChat, and other enterprise communication systems, making it a flexible solution... -
9
ChatGLM2-6B
ChatGLM2-6B: An Open Bilingual Chat LLM
ChatGLM2-6B is the second-gen Chinese-English conversational LLM from ZhipuAI/Tsinghua. It upgrades the base model with GLM’s hybrid pretraining objective, 1.4 TB bilingual data, and preference alignment—delivering big gains on MMLU, CEval, GSM8K, and BBH. The context window extends up to 32K (FlashAttention), and Multi-Query Attention improves speed and memory use. The repo includes Python APIs, CLI & web demos, OpenAI-style/FASTAPI servers, and quantized checkpoints for lightweight local... -
Secure File Transfer for Windows with Cerberus by RedwoodCerberus supports unlimited users and connections on a single IP, with built-in encryption, 2FA, and a browser-based web client — all deployable in under 15 minutes with a 25-day free trial.
-
10
LLM-Pruner
On the Structural Pruning of Large Language Models
LLM-Pruner is an open-source framework designed to compress large language models through structured pruning techniques while maintaining their general capabilities. Large language models often require enormous computational resources, making them expensive to deploy and inefficient for many practical applications. LLM-Pruner addresses this issue by identifying and removing non-essential components within transformer architectures, such as redundant attention heads or feed-forward... -
11
MaxText
A simple, performant and scalable Jax LLM
MaxText is a high-performance, highly scalable open-source framework designed to train and fine-tune large language models using the JAX ecosystem. The project acts as both a reference implementation and a practical training library that demonstrates best practices for building and scaling transformer-based language models on modern accelerator hardware. It is optimized to run efficiently on Google Cloud TPUs and GPUs, enabling researchers and engineers to train models ranging from small... -
12
Lagent
A lightweight framework for building LLM-based agents
Lagent is a lightweight open-source framework designed to help developers build autonomous agents powered by large language models. The framework provides tools and abstractions that allow language models to interact with external tools, execute tasks, and perform multi-step reasoning processes. Instead of using LLMs only for text generation, Lagent enables developers to transform models into agents capable of performing actions such as retrieving data, executing code, or interacting with... -
13
II Agent
A new open-source framework to build and deploy intelligent agents
II-Agent is an open-source intelligent assistant framework designed to automate complex workflows across multiple domains using large language models and external tools. The platform allows users to interact with multiple AI models within a single environment while connecting those models to external services and knowledge sources. Through a unified interface, users can switch between models, access specialized tools, and execute tasks that require information retrieval, code execution, or... -
14
Chitu
High-performance inference framework for large language models
Chitu is a high-performance inference engine designed to deploy and run large language models efficiently in production environments. The framework focuses on improving efficiency, flexibility, and scalability for organizations that need to run LLM inference workloads across different hardware platforms. It supports heterogeneous computing environments, including CPUs, GPUs, and various specialized AI accelerators, allowing models to run across a wide range of infrastructure configurations.... -
15
FastDeploy
High-performance Inference and Deployment Toolkit for LLMs and VLMs
FastDeploy is an open-source inference and deployment toolkit designed to simplify the process of running and serving deep learning models across a wide range of hardware platforms. Developed within the PaddlePaddle ecosystem, the toolkit focuses on providing high-performance deployment capabilities for modern AI models including large language models and vision-language systems. The platform enables developers to deploy trained models quickly using optimized inference pipelines that support... -
16
handy-ollama
Implement CPU from scratch and play with large model deployments
handy-ollama is an open-source educational project designed to help developers and AI enthusiasts learn how to deploy and run large language models locally using the Ollama platform. The repository serves as a structured tutorial that explains how to install, configure, and use Ollama to run modern language models on personal hardware without requiring advanced infrastructure. A key focus of the project is enabling users to run large models even without GPUs by leveraging optimized CPU-based... -
17
NVIDIA Generative AI Examples
Generative AI reference workflows
NVIDIA GenerativeAIExamples is an open-source repository that provides practical reference implementations and example workflows for building generative AI applications using NVIDIA’s software ecosystem. The project is designed to help developers accelerate the development of AI applications by providing ready-to-run pipelines, notebooks, and tools that demonstrate how to integrate large language models into real-world systems. The repository includes examples covering topics such as... -
18
LLM Foundry
LLM training code for MosaicML foundation models
Introducing MPT-7B, the first entry in our MosaicML Foundation Series. MPT-7B is a transformer trained from scratch on 1T tokens of text and code. It is open source, available for commercial use, and matches the quality of LLaMA-7B. MPT-7B was trained on the MosaicML platform in 9.5 days with zero human intervention at a cost of ~$200k. Large language models (LLMs) are changing the world, but for those outside well-resourced industry labs, it can be extremely difficult to train and deploy... -
19
Chinese-LLaMA-Alpaca 2
Chinese LLaMA-2 & Alpaca-2 Large Model Phase II Project
This project is developed based on the commercially available large model Llama-2 released by Meta. It is the second phase of the Chinese LLaMA&Alpaca large model project. The Chinese LLaMA-2 base model and the Alpaca-2 instruction fine-tuning large model are open-sourced. These models expand and optimize the Chinese vocabulary on the basis of the original Llama-2, use large-scale Chinese data for incremental pre-training, and further improve the basic semantics and command understanding of... -
20
YAYI
Repo for YaYi Chinese LLMs based on LlaMA2 & BLOOM
YAYI is an open-source large language model project developed to provide a multilingual conversational AI system capable of performing a wide variety of natural language processing tasks. The model is trained on diverse datasets covering multiple languages and domains so that it can support applications ranging from dialogue systems to text analysis and knowledge retrieval. The architecture is based on transformer-style language models optimized for conversational understanding and... -
21
Aviary
Ray Aviary - evaluate multiple LLMs easily
Aviary is an LLM serving solution that makes it easy to deploy and manage a variety of open source LLMs. Providing an extensive suite of pre-configured open source LLMs, with defaults that work out of the box. Supporting Transformer models hosted on Hugging Face Hub or present on local disk. Aviary has native support for autoscaling and multi-node deployments thanks to Ray and Ray Serve. Aviary can scale to zero and create new model replicas (each composed of multiple GPU workers) in... -
22
LangChain Apps on Production with Jina
Langchain Apps on Production with Jina & FastAPI
Jina is an open-source framework for building scalable multi-modal AI apps on Production. LangChain is another open-source framework for building applications powered by LLMs. long-chain-serve helps you deploy your LangChain apps on Jina AI Cloud in a matter of seconds. You can benefit from the scalability and serverless architecture of the cloud without sacrificing the ease and convenience of local development. And if you prefer, you can also deploy your LangChain apps on your own... -
23
Chinese-LLaMA-Alpaca-2 v2.0
Chinese LLaMA & Alpaca large language model + local CPU/GPU training
This project has open-sourced the Chinese LLaMA model and the Alpaca large model with instruction fine-tuning to further promote the open research of large models in the Chinese NLP community. Based on the original LLaMA , these models expand the Chinese vocabulary and use Chinese data for secondary pre-training, which further improves the basic semantic understanding of Chinese. At the same time, the Chinese Alpaca model further uses Chinese instruction data for fine-tuning, which...
- Previous
- You're on page 1
- Next
