llama.go is like llama.cpp in pure Golang. The code of the project is based on the legendary ggml.cpp framework of Georgi Gerganov written in C++ with the same attitude to performance and elegance. Both models store FP32 weights, so you'll needs at least 32Gb of RAM (not VRAM or GPU RAM) for LLaMA-7B. Double to 64Gb for LLaMA-13B.

Features

  • Tensor math in pure Golang
  • Implement LLaMA neural net architecture and model loading
  • Test with smaller LLaMA-7B model
  • Be sure Go inference works exactly same way as C++
  • Let Go shine! Enable multi-threading and messaging to boost performance
  • Cross-patform compatibility with Mac, Linux and Windows

Project Samples

LLaMA.go Screenshot 1

Project Activity

See All Activity >

Categories

Large Language Models (LLM), AI Models, LLM Inference

License

MIT License

Follow LLaMA.go

LLaMA.go Web Site

Other Useful Business Software
$300 Free Credits to Build on Google Cloud Icon
$300 Free Credits to Build on Google Cloud

New customers can spin up VMs, build with AI, and query data at no cost.

Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
Start Free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of LLaMA.go!

Additional Project Details

Programming Language

Go

Related Categories

Go Large Language Models (LLM), Go AI Models, Go LLM Inference Tool

Registered

2023-08-25
Report inappropriate content