Search Results for "llama.cpp"

The llama.cpp project enables the inference of Meta's LLaMA model (and other models) in pure C/C++ without requiring a Python runtime. It is designed for efficient and fast model execution, offering easy integration for applications needing LLM-based capabilities. The repository focuses on providing a highly optimized and portable implementation for running large language models directly within C/C++ environments.

Hy-MT2 is a family of fast-thinking multilingual translation models built for complex real-world translation scenarios. It includes 1.8B, 7B, and 30B-A3B model sizes, giving users options for lightweight deployment, stronger general performance, or MoE-based capacity. The models support translation across 33 languages and are designed to follow detailed translation instructions in multiple languages. They can handle tasks involving terminology, style, personalization, delimiters, structured...

...Unlike raw base models, it inherits improvements from the SuperGemma Fast line, resulting in better performance in logic, coding, and real-world text workflows. The model is packaged in GGUF format for efficient use with llama.cpp and has been specifically tested on Apple Silicon hardware, delivering high token speeds and smooth local inference. A neutral chat template is embedded to prevent prompt misrouting issues, ensuring consistent responses without unintended shifts into coding or tool-use modes.

...With a context window of 8,192 tokens, it excels at code completion, fill-in-the-middle tasks, and intelligent code suggestions for professional developer tools and IDEs. The model is efficient for both cloud inference with vLLM and local deployment using llama.cpp or Ollama, thanks to its bf16 precision and AMP training. While the base model is not fine-tuned for downstream tasks, it is designed to be easily adapted through supervised fine-tuning (SFT) or reinforcement learning (RL). Benchmarks on RepoBench, SAFIM, and HumanEval demonstrate its competitive performance, with specialized fine-tuned versions for Python already showing strong improvements.

Jan-v1-edge is a lightweight agentic language model developed by JanHQ, designed for fast and reliable on-device execution. It is the second release in the Jan Family and was distilled from the larger Jan-v1 model, retaining strong reasoning and problem-solving capabilities while reducing its computational footprint. The model was refined through a two-stage post-training process: Supervised Fine-Tuning (SFT) to transfer knowledge from Jan-v1, followed by Reinforcement Learning with...