gemma.cpp

Gemma.cpp is a C++ implementation for running inference with Gemma models efficiently on CPUs and GPUs. Developed by Google, it allows running large language models (LLMs) like Gemma with minimal hardware, focusing on optimized performance and low latency. Gemma.cpp is intended for developers seeking to deploy LLMs in production environments without needing massive computational resources.

Features

Efficient C++ implementation for fast inference of Gemma models
Supports both CPU and GPU backends for flexible deployment
Optimized for low-latency and high-performance inference
Lightweight and suitable for resource-constrained environments
Compatible with pre-trained Gemma LLM checkpoints
Easy integration into production pipelines and applications

Project Samples

Project Activity

See All Activity >

License

Apache License V2.0

Follow gemma.cpp

gemma.cpp Web Site

Other Useful Business Software

$300 in Free Credit Towards Top Cloud Services

Build VMs, containers, AI, databases, storage—all in one place.

Start your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.

Get Started

Rate This Project

User Reviews

Be the first to post a review of gemma.cpp!

Additional Project Details

Operating Systems

Linux, Mac, Windows

Programming Language

C++

Related Categories

C++ Artificial Intelligence Software, C++ LLM Inference Tool

Registered

2025-03-13

Similar Business Software

LM-Kit.NET

LM-Kit.NET is a cutting-edge, high-level inference SDK designed specifically to bring the advanced capabilities of Large Language Models (LLM) into the C# ecosystem. Tailored for developers working within .NET, LM-Kit.NET provides a comprehensive suite of powerful Generative AI tools, making...

See Software
Google Compute Engine

Compute Engine is Google's infrastructure as a service (IaaS) platform for organizations to create and run cloud-based virtual machines. Computing infrastructure in predefined or custom machine sizes to accelerate your cloud transformation. General purpose (E2, N1, N2, N2D) machines provide a...

See Software
Vertex AI

Build, deploy, and scale machine learning (ML) models faster, with fully managed ML tools for any use case. Through Vertex AI Workbench, Vertex AI is natively integrated with BigQuery, Dataproc, and Spark. You can use BigQuery ML to create and execute machine learning models in BigQuery...

See Software
RunPod

RunPod offers a cloud-based platform designed for running AI workloads, focusing on providing scalable, on-demand GPU resources to accelerate machine learning (ML) model training and inference. With its diverse selection of powerful GPUs like the NVIDIA A100, RTX 3090, and H100, RunPod supports...

See Software
Google Cloud Platform

Google Cloud is a cloud-based service that allows you to create anything from simple websites to complex applications for businesses of all sizes. New customers get $300 in free credits to run, test, and deploy workloads. All customers can use 25+ products for free, up to monthly usage...

See Software
Pipefy

Pipefy is the AI-driven Business Orchestration and Automation Technologies (BOAT) platform that delivers enterprise results in days, not months. Designed as a secure orchestration layer, Pipefy bridges the gap between rigid legacy systems (ERPs/CRMs) and agile business needs. It allows IT...

See Software