Text Generation Web UI Files

Bug fixes

• Don't use $ $ for LaTeX, only $$ $$, to avoid broken rendering of text like apples cost $1, oranges cost $2
• Fix exllamav3 ignoring the stop button
• Fix a transformers issue when using --bf16 and Flash Attention 2 (#7217). Thanks, @stevenxdavis.
• Fix x86_64 macos portable builds containing arm64 files

Backend updates

• Update llama.cpp to https://github.com/ggml-org/llama.cpp/tree/7f766929ca8e8e01dcceb1c526ee584f7e5e1408
• Update transformers to 4.56

Portable builds

Below you can find self-contained packages that work with GGUF models (llama.cpp) and require no installation! Just download the right version for your system, unzip, and run.

Which version to download:

• Windows/Linux:
• NVIDIA GPU: Use cuda12.4 for newer GPUs or cuda11.7 for older GPUs and systems with older drivers.
• AMD/Intel GPU: Use vulkan builds.

• CPU only: Use cpu builds.

• Mac:

• Apple Silicon: Use macos-arm64.
• Intel CPU: Use macos-x86_64.

Updating a portable install:

1. Download and unzip the latest version.
2. Replace the user_data folder with the one in your existing install. All your settings and models will be moved.