Search Results for "llama-cpp"

...Drop-in replacement for OpenAI running LLMs on consumer-grade hardware. Free Open Source OpenAI alternative. No GPU is required. Runs ggml, GPTQ, onnx, TF compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and many others. LocalAI is a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs (and not only) locally or on-prem with consumer-grade hardware, supporting multiple model families that are compatible with the ggml format. ...

...The code of the project is based on the legendary ggml.cpp framework of Georgi Gerganov written in C++ with the same attitude to performance and elegance. Both models store FP32 weights, so you'll needs at least 32Gb of RAM (not VRAM or GPU RAM) for LLaMA-7B. Double to 64Gb for LLaMA-13B.

...Code is automatically persisted on the file system. You can also use Nodebook directly on the command line, running your notebooks upon change. A notebook is a folder containing an {index|main}.{js,py,c,cpp,...} file. The homepage lists all of the available notebooks. Click on the + Notebook button on the Home page, then select the language of the notebook to be created. Once on the notebook edition page, you can rename the notebook by clicking on it's name. Notebooks are created in the directory specified by the parameter --notebooks.