Search Results for "inference" - Page 4
Sort By:
Showing 110 open source projects for "inference"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
Qwen2.5-Math
A series of math-specific large language models of our Qwen2 series
Qwen2.5-Math is a series of mathematics-specialized large language models in the Qwen2 family, released by Alibaba’s QwenLM. It includes base models (1.5B / 7B / 72B parameters), instruction-tuned versions, and a reward model (RM) to improve alignment. Unlike its predecessor Qwen2-Math, Qwen2.5-Math supports both Chain-of-Thought (CoT) reasoning and Tool-Integrated Reasoning (TIR) for solving math problems, and works in both Chinese and English. It is optimized for solving mathematical... -
2
GLM-4
GLM-4 series: Open Multilingual Multimodal Chat LMs
GLM-4 is a family of open models from ZhipuAI that spans base, chat, and reasoning variants at both 32B and 9B scales, with long-context support and practical local-deployment options. The GLM-4-32B-0414 models are trained on ~15T high-quality data (including substantial synthetic reasoning data), then post-trained with preference alignment, rejection sampling, and reinforcement learning to improve instruction following, coding, function calling, and agent-style behaviors. The... -
3
ReCall
Learning to Reason with Search for LLMs via Reinforcement Learning
ReCall is an open-source framework designed to train and evaluate language models that can reason through complex problems by interacting with external tools. The project builds on earlier work focused on teaching models how to search for information during reasoning tasks and extends that idea to a broader system where models can call a variety of external tools such as APIs, databases, or computation engines. Instead of relying purely on static knowledge stored inside the model, ReCall... -
4
Pixeltable
Data Infrastructure providing an approach to multimodal AI workloads
Pixeltable is an open-source Python data infrastructure framework designed to support the development of multimodal AI applications. The system provides a declarative interface for managing the entire lifecycle of AI data pipelines, including storage, transformation, indexing, retrieval, and orchestration of datasets. Unlike traditional architectures that require multiple tools such as databases, vector stores, and workflow orchestrators, Pixeltable unifies these functions within a... -
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
5
LlamaGen
Autoregressive Model Beats Diffusion
LlamaGen is an open-source research project that introduces a new approach to image generation by applying the autoregressive next-token prediction paradigm used in large language models to visual generation tasks. Instead of relying on diffusion models, the framework treats images as sequences of tokens that can be generated progressively using transformer architectures similar to those used for text generation. The project explores how scaling autoregressive models and improving image... -
6
xLSTM
Neural Network architecture based on ideas of the original LSTM
...The architecture aims to provide competitive performance with transformer-based models while maintaining advantages such as linear computational scaling and efficient memory usage for long sequences. Researchers have demonstrated that xLSTM models can scale to billions of parameters and large training datasets while maintaining efficient inference speeds. -
7
TigerBot
TigerBot: A multi-language multi-task LLM
...The project provides both base models and chat-optimized variants that can be used for dialogue systems, question answering, and general language understanding tasks. In addition to model weights, the repository includes training scripts, inference tools, and configuration files that allow researchers and developers to reproduce experiments or fine-tune the models for specific applications. -
8
Chat with LLMs Everywhere
Run PyTorch LLMs locally on servers, desktop and mobile
TorchChat is an open-source project from the PyTorch ecosystem designed to demonstrate how large language models can be executed efficiently across different computing environments. The project provides a compact codebase that illustrates how to run conversational AI systems using PyTorch models on laptops, servers, and mobile devices. It is intended primarily as a reference implementation that shows developers how to integrate large language models into applications without requiring a... -
9
StarVector
StarVector is a foundation model for SVG generation
StarVector is a multimodal foundation model designed for generating Scalable Vector Graphics (SVG) from images or textual descriptions. The system treats vector graphics creation as a code generation problem, producing SVG code that can render detailed vector images. Its architecture combines computer vision techniques with language modeling capabilities so it can understand visual inputs and textual prompts simultaneously. The model converts raster images or text instructions into... -
Go From AI Idea to AI App FastAccess Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
-
10
handy-ollama
Implement CPU from scratch and play with large model deployments
...The repository serves as a structured tutorial that explains how to install, configure, and use Ollama to run modern language models on personal hardware without requiring advanced infrastructure. A key focus of the project is enabling users to run large models even without GPUs by leveraging optimized CPU-based inference pipelines. The project includes step-by-step guides that walk learners through tasks such as installing Ollama, managing local models, calling model APIs, and building simple AI applications on top of locally hosted models. Through hands-on exercises and practical examples, the tutorial demonstrates how developers can create applications like chat assistants or retrieval systems using locally deployed models. -
11
VibeThinker
Diversity-driven optimization and large-model reasoning ability
VibeThinker is a compact but high-capability open-source language model released by WeiboAI (Sina AI Lab). It contains about 1.5 billion parameters, far smaller than many “frontier” models, yet it is explicitly optimized for reasoning, mathematics, and code generation tasks rather than general open-domain chat. The innovation lies in its training methodology: the team uses what they call the Spectrum-to-Signal Principle (SSP), where a first stage emphasizes diversity of reasoning paths (the... -
12
local-llm
Run LLMs locally on Cloud Workstations
...The repository includes tools, Docker configurations, and command-line utilities that simplify the process of downloading, running, and interacting with language models directly on local or cloud-based workstations. This approach improves data privacy and control, as all inference can be performed locally without sending sensitive information to external APIs. It also integrates seamlessly with Google Cloud services, allowing developers to build and test AI-powered applications within the broader cloud ecosystem. -
13
RAG-Retrieval
Unify Efficient Fine-tuning of RAG Retrieval, including Embedding
...Retrieval-augmented generation combines large language models with external knowledge retrieval to improve factual accuracy and domain-specific reasoning. This repository provides end-to-end infrastructure for training retrieval models, performing inference, and distilling embedding models for improved performance. It includes implementations of modern embedding architectures designed to map documents and queries into vector spaces for efficient similarity search. The framework also supports reranking models that refine retrieved results using large language models or lightweight transformer architectures. ... -
14
Bard API
The unofficial python package that returns response of Google Bard
The Python package returns a response of Google Bard through the value of the cookie. This package is designed for application to the Python package ExceptNotifier and Co-Coder. Please note that the bardapi is not a free service, but rather a tool provided to assist developers with testing certain functionalities due to the delayed development and release of Google Bard's API. It has been designed with a lightweight structure that can easily adapt to the emergence of an official API.... -
15
LLaVA
Visual Instruction Tuning: Large Language-and-Vision Assistant
Visual instruction tuning towards large language and vision models with GPT-4 level capabilities. -
16
Mixtral offloading
Run Mixtral-8x7B models in Colab or consumer desktops
Mixtral-Offloading is an open-source project designed to enable efficient inference of large Mixture-of-Experts language models such as Mixtral-8x7B on hardware with limited GPU memory. The project implements techniques that allow model components to be dynamically moved between CPU memory and GPU memory during inference, significantly reducing the amount of GPU VRAM required to run the model. This approach takes advantage of the sparse activation properties of mixture-of-experts architectures, where only a subset of expert networks are used for each token during generation. ... -
17
Grok-1
Open-source, high-performance Mixture-of-Experts large language model
Grok-1 is a 314-billion-parameter Mixture-of-Experts (MoE) large language model developed by xAI. Designed to optimize computational efficiency, it activates only 25% of its weights for each input token. In March 2024, xAI released Grok-1's model weights and architecture under the Apache 2.0 license, making them openly accessible to developers. The accompanying GitHub repository provides JAX example code for loading and running the model. Due to its substantial size, utilizing Grok-1... -
18
YAYI
Repo for YaYi Chinese LLMs based on LlaMA2 & BLOOM
...In addition to producing coherent responses, the system is designed to handle tasks such as summarization, translation, question answering, and text classification. The repository provides model checkpoints, training resources, and inference tools that allow developers to deploy the model in their own applications. By releasing both the model and supporting infrastructure, the project encourages experimentation and research in multilingual AI systems. -
19
Punica
Serving multiple LoRA finetuned LLM as one
...LoRA is a parameter-efficient fine-tuning method that allows developers to adapt large pretrained models to specific tasks by adding lightweight adapter layers rather than retraining the entire model. Punica introduces a serving architecture that allows multiple LoRA adapters to share the same base model during inference, significantly reducing memory consumption and computational overhead. The system includes specialized CUDA kernels that enable batched GPU operations across different LoRA models simultaneously. This design allows a single GPU cluster to host many task-specific models while maintaining high throughput and minimal latency. The architecture also includes scheduling mechanisms that coordinate requests from multiple tenants and distribute workloads efficiently across available resources. -
20
RAGxplorer
Open-source tool to visualise your RAG
RAGxplorer is an open-source visualization tool designed to help developers analyze and understand Retrieval-Augmented Generation (RAG) pipelines. Retrieval-augmented generation combines language models with external document retrieval systems in order to produce more accurate and grounded responses. However, RAG systems can be complex because they involve multiple components such as embedding models, vector databases, and retrieval algorithms. RAGxplorer provides visual tools that allow... -
21
InternLM
Official release of InternLM series
InternLM is an open-source family of multilingual foundation and chat models, accompanied by an ecosystem that supports training, inference, and application development. The repository highlights multiple model sizes intended to serve different needs, from efficient research and prototyping to more capable deployments for complex scenarios. Beyond model weights, the project emphasizes an ecosystem view, pointing developers to compatible tools and projects across training and inference so teams can build end-to-end workflows. ... -
22
towhee
Framework that is dedicated to making neural data processing
...From images to text to 3D molecular structures, Towhee supports data transformation for nearly 20 different unstructured data modalities. We provide end-to-end pipeline optimizations, covering everything from data decoding/encoding, to model inference, making your pipeline execution 10x faster. Towhee provides out-of-the-box integration with your favorite libraries, tools, and frameworks, making development quick and easy. Towhee includes a pythonic method-chaining API for describing custom data processing pipelines. We also support schemas, making processing unstructured data as easy as handling tabular data. -
23
LLaMA-MoE
Building Mixture-of-Experts from LLaMA with Continual Pre-training
...Its architecture works by splitting LLaMA feed-forward networks into sparse experts and adding gating mechanisms so that only selected experts are activated during inference and training. The project is not just a model release, but also a research framework that includes multiple expert construction methods, several gating strategies, and tooling for continual pre-training on filtered SlimPajama-based datasets. It also emphasizes training efficiency through features such as FlashAttention-v2 integration and fast streaming dataset loading, which are important for large-scale experimentation. -
24
EvaDB
Database system for building simpler and faster AI-powered application
...For example, the state-of-the-art object detection model takes multiple GPU years to process just a week’s videos from a single traffic monitoring camera. Besides the money spent on hardware, these models also increase the time that you spend waiting for the model inference to finish. -
25
DB-GPT-Hub
A repository that contains models, datasets, and fine-tuning
...The project serves as a specialized extension of the broader DB-GPT ecosystem, focusing on the preparation and evaluation of models capable of translating natural language questions into structured database queries. It offers a modular framework that supports data preparation, model fine-tuning, benchmarking, and inference for Text-to-SQL systems. The repository includes datasets and experiment configurations that allow researchers to train models on real database schemas and evaluate them using standardized benchmarks. Its design encourages experimentation with different large language models and fine-tuning techniques, including parameter-efficient training approaches.
