Search Results for "net"

The C#/.NET binding of llama.cpp. It provides APIs to infer the LLaMa Models and deploy it on the local environment. It works on both Windows, Linux and MAC without the requirement for compiling llama.cpp yourself. Its performance is close to llama.cpp. Furthermore, it provides integrations with other projects such as BotSharp to provide higher-level applications and UI.

OllamaSharp is an open-source .NET library that provides strongly typed bindings for interacting with the Ollama API, making it easier for developers to integrate local large language models into C# and .NET applications. The project acts as a wrapper around the Ollama API, exposing all endpoints through asynchronous methods that allow developers to perform tasks such as generating text, creating embeddings, and managing models.

llama.go is like llama.cpp in pure Golang. The code of the project is based on the legendary ggml.cpp framework of Georgi Gerganov written in C++ with the same attitude to performance and elegance. Both models store FP32 weights, so you'll needs at least 32Gb of RAM (not VRAM or GPU RAM) for LLaMA-7B. Double to 64Gb for LLaMA-13B.