Running Llama 2 with gradio web UI on GPU or CPU from anywhere (Linux/Windows/Mac).

Features

  • Supporting all Llama 2 models (7B, 13B, 70B, GPTQ, GGML) with 8-bit, 4-bit mode
  • Use llama2-wrapper as your local llama2 backend for Generative Agents/Apps; colab example
  • Run OpenAI Compatible API on Llama2 models
  • Supporting models: Llama-2-7b/13b/70b, all Llama-2-GPTQ, all Llama-2-GGML
  • Supporting model backends: tranformers, bitsandbytes(8-bit inference), AutoGPTQ(4-bit inference), llama.cpp
  • Demos: Run Llama2 on MacBook Air; Run Llama2 on free Colab T4 GPU

Project Samples

llama2-webui Screenshot 1

Project Activity

See All Activity >

Categories

Large Language Models (LLM), LLM Inference

License

MIT License

Follow llama2-webui

llama2-webui Web Site

Other Useful Business Software
Go from Code to Production URL in Seconds Icon
Go from Code to Production URL in Seconds

Cloud Run deploys apps in any language instantly. Scales to zero. Pay only when code runs.

Skip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
Try it free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of llama2-webui!

Additional Project Details

Programming Language

Python

Related Categories

Python Large Language Models (LLM), Python LLM Inference Tool

Registered

2023-08-25
Report inappropriate content