Running Llama 2 with gradio web UI on GPU or CPU from anywhere (Linux/Windows/Mac).

Features

  • Supporting all Llama 2 models (7B, 13B, 70B, GPTQ, GGML) with 8-bit, 4-bit mode
  • Use llama2-wrapper as your local llama2 backend for Generative Agents/Apps; colab example
  • Run OpenAI Compatible API on Llama2 models
  • Supporting models: Llama-2-7b/13b/70b, all Llama-2-GPTQ, all Llama-2-GGML
  • Supporting model backends: tranformers, bitsandbytes(8-bit inference), AutoGPTQ(4-bit inference), llama.cpp
  • Demos: Run Llama2 on MacBook Air; Run Llama2 on free Colab T4 GPU

Project Samples

llama2-webui Screenshot 1

Project Activity

See All Activity >

Categories

Large Language Models (LLM), LLM Inference

License

MIT License

Follow llama2-webui

llama2-webui Web Site

Other Useful Business Software
$300 Free Credits to Build on Google Cloud Icon
$300 Free Credits to Build on Google Cloud

New customers can spin up VMs, build with AI, and query data at no cost.

Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
Start Free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of llama2-webui!

Additional Project Details

Programming Language

Python

Related Categories

Python Large Language Models (LLM), Python LLM Inference Tool

Registered

2023-08-25
Report inappropriate content