Running Llama 2 with gradio web UI on GPU or CPU from anywhere (Linux/Windows/Mac).

Features

  • Supporting all Llama 2 models (7B, 13B, 70B, GPTQ, GGML) with 8-bit, 4-bit mode
  • Use llama2-wrapper as your local llama2 backend for Generative Agents/Apps; colab example
  • Run OpenAI Compatible API on Llama2 models
  • Supporting models: Llama-2-7b/13b/70b, all Llama-2-GPTQ, all Llama-2-GGML
  • Supporting model backends: tranformers, bitsandbytes(8-bit inference), AutoGPTQ(4-bit inference), llama.cpp
  • Demos: Run Llama2 on MacBook Air; Run Llama2 on free Colab T4 GPU

Project Samples

llama2-webui Screenshot 1

Project Activity

See All Activity >

Categories

Large Language Models (LLM), LLM Inference

License

MIT License

Follow llama2-webui

llama2-webui Web Site

Other Useful Business Software
Custom VMs From 1 to 96 vCPUs With 99.95% Uptime Icon
Custom VMs From 1 to 96 vCPUs With 99.95% Uptime

General-purpose, compute-optimized, or GPU/TPU-accelerated. Built to your exact specs.

Live migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
Try Free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of llama2-webui!

Additional Project Details

Programming Language

Python

Related Categories

Python Large Language Models (LLM), Python LLM Inference Tool

Registered

2023-08-25
Report inappropriate content