Download node-llama-cpp-electron-example.macOS.3.20.0.arm64.zip (node-llama-cpp)

Other Useful Business Software

Build Securely on AWS with Proven Frameworks

Lay a foundation for success with Tested Reference Architectures developed by Fortinet’s experts. Learn more in this white paper.

Moving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.

Download Now

Save Up to 91% on Cloud Compute With Spot VMs

Automatic sustained-use discounts. One free VM per month. No negotiation needed.

Run batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.

Try Free

Host LLMs in Production With On-Demand GPUs

NVIDIA L4 GPUs. 5-second cold starts. Scale to zero when idle.

Deploy your model, get an endpoint, pay only for compute time. No GPU provisioning or infrastructure management required.

Try Free

$300 Free Credits to Build on Google Cloud

New customers can spin up VMs, build with AI, and query data at no cost.

Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.

MongoDB Atlas runs apps anywhere

Deploy in 115+ regions with the modern database for every enterprise.

MongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.

Start Free

node-llama-cpp Overview

Run AI models locally on your machine with node.js bindings for llama

node-llama-cpp is a JavaScript and Node.js binding that allows developers to run large language models locally using the high-performance inference engine provided by llama.cpp. The library enables applications built with Node.js to interact directly with local LLM models without requiring a remote API or external service. By using native bindings and optimized model execution, the framework allows developers to integrate advanced language model capabilities into desktop applications, server software, and command-line tools. The system automatically detects the available hardware on a machine and selects the most appropriate compute backend, including CPU or GPU acceleration. Developers can use the library to perform tasks such as text generation, conversational chat, embedding generation, and structured output generation. Because it runs models locally, the platform is particularly useful for privacy-sensitive environments or offline AI deployments.

Additional Details for node-llama-cpp

Operating Systems

Linux, Mac, Windows

Programming Language

TypeScript

Registered

2026-03-06

Last Updated

6 days ago

Maintainers

sf-editor1

Keep Me Updated!

Get an email when this software releases a new version (no other emails will be sent). Get newsletters and notices that include site news, special offers and exclusive discounts about IT products & services.

I understand by clicking on "Subscribe" below I am agreeing to the SourceForge Terms of Use and the Privacy Policy which describe how we use and share your data. I agree to receive these communications from SourceForge.net. I understand that I can withdraw my consent at anytime.

JavaScript is required for this form.

node-llama-cpp