Open Source C++ Generative AI

The llama.cpp project enables the inference of Meta's LLaMA model (and other models) in pure C/C++ without requiring a Python runtime. It is designed for efficient and fast model execution, offering easy integration for applications needing LLM-based capabilities. The repository focuses on providing a highly optimized and portable implementation for running large language models directly within C/C++ environments.

Gnoppix is a Linux distribution based on Debian Linux available in for amd64 and ARM architectures. Gnoppix is a great choice for users who want a lightweight and easy-to-use with security in mind. Gnoppix was first announced in June 2003. Currently we're working on a Gnoppix version for WSL, Mobile devices like smartphones and tablets as well.

KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models, inspired by the original KoboldAI. It's a single self-contained distributable that builds off llama.cpp and adds many additional powerful features.

Run a fast ChatGPT-like model locally on your device. This combines the LLaMA foundation model with an open reproduction of Stanford Alpaca a fine-tuning of the base model to obey instructions (akin to the RLHF used to train ChatGPT) and a set of modifications to llama.cpp to add a chat interface. Download the zip file corresponding to your operating system from the latest release. The weights are based on the published fine-tunes from alpaca-lora, converted back into a PyTorch checkpoint with a modified script and then quantized with llama.cpp the regular way.

C++ Implementation of PyTorch Tutorials for Everyone. This repository provides tutorial code in C++ for deep learning researchers to learn PyTorch (i.e. Section 1 to 3) Interactive Tutorials are currently running on LibTorch Nightly Version. Libtorch only supports 64bit Windows and an x64 generator needs to be specified. Create all required script module files for pre-learned models/weights during the build. Requires installed python3 with PyTorch and torch-vision. You can choose to only build tutorials in one of the categories basics, intermediate, advanced or popular. You can build and run the tutorials (on CPU) in a Docker container using the provided Dockerfile and docker-compose.yml files.

DomE Experiment is an implementation of a reference architecture for creating information systems from the automated evolution of the domain model. The architecture comprises elements that guarantee user access through automatically generated interfaces for various devices, integration with external information sources, data and operations security, automatic generation of analytical information, and automatic control of business processes. All these features are generated from the domain model, which is, in turn, continuously evolved from interactions with the user or autonomously by the system itself. Thus, an alternative to the traditional software production processes is proposed, which involves several stages and different actors, sometimes demanding a lot of time and money without obtaining the expected result. With software engineering techniques, self-adaptive systems, and artificial intelligence, it is possible, the integration between design time and execution time.

RETRO (Retrieval-Enhanced Transformer) is a large language model architecture developed by OpenAI that augments transformer models with a retrieval mechanism. Instead of relying solely on learned parameters, RETRO retrieves relevant documents from a large external database during inference, allowing it to ground responses in external knowledge. This design improves factual accuracy, reduces hallucinations, and enables smaller models to perform comparably to much larger ones by leveraging retrieval. The repository provides code and resources for training and evaluating RETRO models, along with infrastructure for integrating retrieval into the transformer pipeline. It includes example configurations, datasets, and utilities for building retrieval-augmented generation systems. RETRO represents an important step toward combining large-scale language modeling with information retrieval, offering researchers a foundation to study hybrid approaches for scaling AI responsibly.