GPU builds of the llama-rpc image (CUDA, Vulkan)
Pool the spare power of every computer into one cloud. Two apps (sys0
Brought to you by:
a-hackers-guide
Menu
▾
▴
#28 GPU builds of the llama-rpc image (CUDA, Vulkan)
Originally created by: A-Hackers-Guide
ghcr.io/a-hackers-guide/llama-rpc is CPU-only today. Add CUDA and Vulkan variants (tags like b12019-cuda, b12019-vulkan) built in .github/workflows/llama-rpc.yml, and let llama-distributed pick the variant per machine from its GPU inventory. Mark GPU workers with "gpu": true once the image can use them.
