Search Results for "device query"

MobileLLM is a lightweight large language model (LLM) framework developed by Facebook Research, optimized for on-device deployment where computational and memory efficiency are critical. Introduced in the ICML 2024 paper “MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases”, it focuses on delivering strong reasoning and generalization capabilities in models under one billion parameters. The framework integrates several architectural innovations—SwiGLU activation, deep and thin network design, embedding sharing, and grouped-query attention (GQA)—to achieve a superior trade-off between model size, inference speed, and accuracy. ...

LocalGPT is a private, on-premises document intelligence platform for questioning, summarizing, and analyzing files with locally hosted language models. Its data remains on the user’s machine, making it suitable for confidential or offline workflows. The retrieval system combines semantic similarity, keyword matching, late chunking, contextual enrichment, and sentence-level pruning. A smart router chooses between retrieval-augmented generation and direct model responses for each query. An...

MiniCPM is an open-source family of compact large language models developed by OpenBMB for efficient AI inference on personal devices and resource-constrained hardware. Its latest MiniCPM5 series includes 1B and 2B-class models optimized for local deployment. The models support text generation, coding, mathematical reasoning, and autonomous agent workflows. MiniCPM5-2B uses a Transformer architecture with grouped-query attention and a 131,072-token context window. Earlier releases explore...