Open Source C Generative AI

Inference of LLaMA model in pure C/C++

Run a fast ChatGPT-like model locally on your device. This combines the LLaMA foundation model with an open reproduction of Stanford Alpaca a fine-tuning of the base model to obey instructions (akin to the RLHF used to train ChatGPT) and a set of modifications to llama.cpp to add a chat interface. Download the zip file corresponding to your operating system from the latest release. The weights are based on the published fine-tunes from alpaca-lora, converted back into a PyTorch checkpoint with a modified script and then quantized with llama.cpp the regular way.

Grenade is a composable, dependently typed, practical, and fast recurrent neural network library for concise and precise specifications of complex networks in Haskell. Because the types are so rich, there's no specific term level code required to construct this network; although it is of course possible and easy to construct and deconstruct the networks and layers explicitly oneself. Networks in Grenade can be thought of as a heterogeneous list of layers, where their type includes not only the layers of the network but also the shapes of data that are passed between the layers. To perform back propagation, one can call the eponymous function which takes a network, appropriate input, and target data, and returns the back propagated gradients for the network. The shapes of the gradients are appropriate for each layer and may be trivial for layers like Relu which have no learnable parameters.