llama.go is like llama.cpp in pure Golang. The code of the project is based on the legendary ggml.cpp framework of Georgi Gerganov written in C++ with the same attitude to performance and elegance. Both models store FP32 weights, so you'll needs at least 32Gb of RAM (not VRAM or GPU RAM) for LLaMA-7B. Double to 64Gb for LLaMA-13B.

Features

  • Tensor math in pure Golang
  • Implement LLaMA neural net architecture and model loading
  • Test with smaller LLaMA-7B model
  • Be sure Go inference works exactly same way as C++
  • Let Go shine! Enable multi-threading and messaging to boost performance
  • Cross-patform compatibility with Mac, Linux and Windows

Project Samples

LLaMA.go Screenshot 1

Project Activity

See All Activity >

Categories

Large Language Models (LLM), AI Models, LLM Inference

License

MIT License

Follow LLaMA.go

LLaMA.go Web Site

Other Useful Business Software
Auth0 B2B Essentials: SSO, MFA, and RBAC Built In Icon
Auth0 B2B Essentials: SSO, MFA, and RBAC Built In

Unlimited organizations, 3 enterprise SSO connections, role-based access control, and pro MFA included. Dev and prod tenants out of the box.

Auth0's B2B Essentials plan gives you everything you need to ship secure multi-tenant apps. Unlimited orgs, enterprise SSO, RBAC, audit log streaming, and higher auth and API limits included. Add on M2M tokens, enterprise MFA, or additional SSO connections as you scale.
Sign Up Free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of LLaMA.go!

Additional Project Details

Programming Language

Go

Related Categories

Go Large Language Models (LLM), Go AI Models, Go LLM Inference Tool

Registered

2023-08-25
Report inappropriate content