Search Results for "multimedia question answering"
Sort By:
Showing 145 open source projects for "multimedia question answering"
View related business solutions-
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
Full-stack observability with actually useful AI | Grafana CloudBuilt on open standards like Prometheus and OpenTelemetry, Grafana Cloud includes Kubernetes Monitoring, Application Observability, Incident Response, plus the AI-powered Grafana Assistant. Get started with our generous free tier today.
-
1
WebGLM
An Efficient Web-enhanced Question Answering System
WebGLM is a web-enhanced question-answering system that combines a large language model with web search and retrieval capabilities to produce more accurate answers. The system is based on the General Language Model architecture and was designed to enable language models to interact directly with web information during the question-answering process. Instead of relying solely on knowledge stored in the model’s training data, the system retrieves relevant web content and integrates it into the reasoning process. ... -
2
DriveLM
Driving with Graph Visual Question Answering
DriveLM is a research-oriented framework and dataset designed to explore how vision-language models can be integrated into autonomous driving systems. The project introduces a new paradigm called graph visual question answering that structures reasoning about driving scenes through interconnected tasks such as perception, prediction, planning, and motion control. Instead of treating autonomous driving as a purely sensor-driven pipeline, DriveLM frames it as a reasoning problem where models answer structured questions about the environment to guide decision making. ... -
3
Kimi-Audio
Audio foundation model excelling in audio understanding
Kimi-Audio is an ambitious open-source audio foundation model designed to unify a wide array of audio processing tasks — from speech recognition and audio understanding to generative conversation and sound event classification — within a single cohesive architecture. Instead of fragmenting work across specialized models, Kimi-Audio handles automatic speech recognition (ASR), audio question answering, automatic audio captioning, speech emotion recognition, and audio-to-text chat in one system, enabling developers to build rich, multimodal audio applications without stitching together disparate components. It uses a novel model setup that combines continuous acoustic features with discrete semantic tokens to richly capture sound and meaning across speech, music, and environmental audio. -
4
Cognita
Open source RAG framework for building scalable modular AI apps
...Cognita provides reusable components such as parsers, data loaders, embedders, retrievers, and query controllers, allowing teams to customize each stage of the RAG pipeline independently. It includes both a backend service and a frontend interface, enabling users to upload documents, experiment with configurations, and perform question-answering tasks interactively. Cognita supports incremental indexing, meaning it processes only new or updated data to reduce computational overhead and improve efficiency. -
Go From AI Idea to AI App FastAccess Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
-
5
MedGemma
Collection of Gemma 3 variants that are trained for performance
...It includes multiple variants such as a 4 billion-parameter multimodal model that can process both medical images and text and a 27 billion-parameter text-only (and multimodal) model that offers deeper clinical reasoning and understanding at higher capacity, making it suitable for complex tasks like medical question answering, summarization of clinical notes, or generating reports from radiology images. The multimodal versions pair a SigLIP-based image encoder pre-trained on diverse de-identified medical imaging data. -
6
PaperQA2
High accuracy RAG for answering questions from scientific documents
PaperQA2 is a package for doing high-accuracy retrieval augmented generation (RAG) on PDFs or text files, with a focus on the scientific literature. See our recent 2024 paper to see examples of PaperQA2's superhuman performance in scientific tasks like question answering, summarization, and contradiction detection. In this example we take a folder of research paper PDFs, magically get their metadata - including citation counts and a retraction check, then parse and cache PDFs into a full-text search index, and finally answer the user question with an LLM agent. -
7
RAG Web UI
RAG Web UI is an intelligent dialogue system based on RAG
RAG Web UI is an open-source intelligent dialogue system built on retrieval-augmented generation technology, designed to enable users to create AI-powered question answering systems grounded in their own knowledge bases. It combines document retrieval with large language models to provide accurate, context-aware responses based on indexed data rather than generic model knowledge. The platform supports ingestion of multiple document formats, including PDFs, Word files, Markdown, and plain text, automatically processing and vectorizing them for efficient retrieval. ... -
8
OP Vault
Give ChatGPT long-term memory using the OP Stack
...It combines a backend written in Go with a React frontend, allowing users to upload files such as PDFs, text documents, and books to create a searchable repository of information. The system uses vector databases like Pinecone alongside OpenAI models to index and retrieve relevant content, enabling precise question-answering grounded in the uploaded materials. Users can query the system in natural language and receive answers that include references to specific files and sections, improving transparency and trust in the responses. The project is designed to handle large volumes of data, making it suitable for personal knowledge management, research archives, or enterprise documentation systems. -
9
ViMax
Director, Screenwriter, Producer, and Video Generator All-in-One
...It integrates components like visual encoders, cross-modal fusion techniques, and reasoning modules so that users can go beyond simple captioning or classification to perform tasks such as visual question answering, multi-image inference, and structured scene understanding. ViMax’s design accommodates large image sets and supports retrieval augmentation, enabling it to work with external image databases, supplementary metadata, and semantic search to enhance context awareness. The system aims to bridge foundational vision backbones and generative language models through adapters and fusion layers that maximize both signal integration and reasoning depth, and includes utility pipelines for training, evaluation, and deployment. -
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
10
TigerBot
TigerBot: A multi-language multi-task LLM
...TigerBot models are based on modern transformer architectures and are trained on large datasets that cover multiple domains and languages. The project provides both base models and chat-optimized variants that can be used for dialogue systems, question answering, and general language understanding tasks. In addition to model weights, the repository includes training scripts, inference tools, and configuration files that allow researchers and developers to reproduce experiments or fine-tune the models for specific applications. -
11
FastGPT
FastGPT is a knowledge-based platform built on the LLMs
FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration. -
12
PandaWiki
AI-powered open source platform for building intelligent wiki bases
...PandaWiki provides tools for managing knowledge bases through an administrative interface while also generating public-facing wiki sites where users can browse and interact with content. AI capabilities are integrated to assist with content creation, intelligent question answering, and semantic search, helping users find information more efficiently within stored documentation. PandaWiki also supports importing knowledge from different external sources such as web pages, RSS feeds, sitemaps, and offline files to quickly populate documentation collections. -
13
MING
A large-scale model of medical consultation in Chinese
MING is an open-source medical large language model designed for intelligent medical consultation and question answering in Chinese. The project focuses on building a healthcare-focused conversational system capable of responding to medical questions, analyzing case descriptions, and guiding diagnostic reasoning. It is trained using medical instruction tuning so that the model can understand patient symptoms and respond with structured explanations and clinical suggestions. ... -
14
Huatuo-Llama-Med-Chinese
Instruction-tuning LLM with Chinese Medical Knowledge
...The project builds specialized models by fine-tuning architectures such as LLaMA, Alpaca-Chinese, and Bloom with curated medical datasets. These datasets are constructed from medical knowledge graphs, academic literature, and question-answer pairs designed to teach models how to respond accurately to healthcare-related queries. The goal of the project is to improve the reliability and domain expertise of language models when answering medical questions or assisting with healthcare-related tasks. By combining domain-specific training data with instruction-tuning techniques, the project produces models capable of generating more accurate medical responses than general-purpose models. -
15
LangChain-ChatGLM-Webui
Automatic question answering for local knowledge bases based on LLM
LangChain-ChatGLM-Webui is an open-source web interface that integrates the ChatGLM large language model with the LangChain framework to create an interactive conversational AI platform. The project provides a graphical interface that allows users to interact with language models through chat sessions while also connecting those models to external knowledge sources. It supports retrieval-augmented generation workflows that enable the system to answer questions based on local documents or... -
16
BEIR
A Heterogeneous Benchmark for Information Retrieval
BEIR is a benchmark framework for evaluating information retrieval models across various datasets and tasks, including document ranking and question answering. -
17
hfapigo
Unofficial (Golang) Go bindings for the Hugging Face Inference API
(Golang) Go bindings for the Hugging Face Inference API. Directly call any model available in the Model Hub. An API key is required for authorized access. To get one, create a Hugging Face profile. -
18
Vidi2
Large Multimodal Models for Video Understanding and Editing
...or “where in the frame is object Y during that moment?” — offering temporal retrieval, spatio-temporal grounding (i.e. locating objects over time + space), and even video question answering. Vidi targets applications like intelligent video editing, automated video search, content analysis, and editing assistance, enabling users to efficiently locate relevant segments and objects in hours-long footage. The system is built with open-source release in mind, giving developers access to model code, inference scripts, and evaluation pipelines so they can reproduce research results or integrate Vidi into their own video-processing workflows. -
19
Haystack
Haystack is an open source NLP framework to interact with your data
Apply the latest NLP technology to your own data with the use of Haystack's pipeline architecture. Implement production-ready semantic search, question answering, summarization and document ranking for a wide range of NLP applications. Evaluate components and fine-tune models. Ask questions in natural language and find granular answers in your documents using the latest QA models with the help of Haystack pipelines. Perform semantic search and retrieve ranked documents according to meaning, not just keywords! ... -
20
RAGFlow
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding. It offers a streamlined RAG workflow for businesses of any scale, combining LLM (Large Language Models) to provide truthful question-answering capabilities, backed by well-founded citations from various complex formatted data. -
21
ChatWiki
ChatWiki WeChat official account's AI knowledge base workflow agent
ChatWiki is an open-source AI knowledge base and workflow automation platform designed to help organizations build intelligent question-answering systems using large language models and retrieval-augmented generation techniques. The system enables companies to transform internal documents and data into searchable knowledge bases that can power AI assistants capable of answering domain-specific questions. It provides a complete pipeline for ingesting documents, preprocessing and segmenting content, generating vector embeddings, and retrieving relevant information during conversations. ... -
22
HuixiangDou
Overcoming Group Chat Scenarios with LLM-based Technical Assistance
HuixiangDou is an open-source large language model assistant designed specifically for technical question answering in group chat environments. The project addresses a common problem in developer communities where discussion channels become overwhelmed by repeated or irrelevant questions. To solve this issue, HuixiangDou implements a multi-stage pipeline that analyzes incoming messages, filters irrelevant conversations, and selectively generates responses when the assistant determines it can provide useful information. ... -
23
Feynman
The open source AI research agent
...It supports advanced workflows like deep research investigations, paper replication, peer review simulation, and autonomous experimentation, enabling users to go beyond simple question answering into full research automation. -
24
CodiumAI PR-Agent
AI-Powered tool for automated pull request analysis
CodiumAI PR-Agent is an open-source tool aiming to help developers review pull requests faster and more efficiently. It automatically analyzes the pull request and can provide several types of commands. See the Usage Guide for instructions how to run the different tools from CLI, online usage, Or by automatically triggering them when a new PR is opened. You can try GPT-4 powered PR-Agent, on your public GitHub repository, instantly. Just mention @CodiumAI-Agent and add the desired command in... -
25
ChatGLM-6B
ChatGLM-6B: An Open Bilingual Dialogue Language Model
...The project provides inference code, demos (command line, web, API), quantization support for lower memory deployment, and tools for finetuning (e.g., via P-Tuning v2). It is optimized for dialogue and question answering with a balance between performance and deployability in consumer hardware settings. Support for quantized inference (INT4, INT8) to reduce GPU memory requirements. Automatic mode switching between precision/memory tradeoffs (full/quantized).
