llama-cpp

Name of the Script

llama-cpp

Script Type

CT (LXC Container)

Does this script support arm64?

arm64 supported

📋 Script Details

llama.cpp runs GGUF models in C/C++ with no Python runtime. llama-server serves a built-in web chat UI and an OpenAI-compatible API on the same port, so it can back AnythingLLM (#2120), Open WebUI or Claude Code. Lighter than the existing ollama script.

• No compiling: prebuild mode against llama-*-bin-ubuntu-$(arch_resolve x64 arm64).tar.gz from the daily release line (currently b10240). Binaries sit flat in the archive, so LD_LIBRARY_PATH=/opt/llama-cpp is set in the unit for the bundled .so files.
• Usable out of the box: configured through LLAMA_ARG_* env vars, which map 1:1 to the CLI flags. LLAMA_ARG_HF_REPO makes llama-server pull a model from Hugging Face on first start (default ggml-org/gemma-3-1b-it-GGUF, ~800 MB) into /opt/llama-cpp_data/models via LLAMA_CACHE. No manual model juggling before the first launch, and LLAMA_ARG_MODEL still works for local files.
• var_gpu="yes" — the release also ships vulkan, rocm and sycl variants; the script installs the CPU build with LLAMA_ARG_N_GPU_LAYERS=0 and documents the switch.
• Note: no authentication by default. LLAMA_ARG_API_KEY is mentioned in the JSON notes as the fix.

Source: https://github.com/ggml-org/llama.cpp

Start by comparing the existing ollama script with the proposed llama-cpp CT script and the llama.cpp release archive named in the issue. Verify the llama-server entry point, LLAMA_ARG_* configuration, arm64 archive selection, model cache path, and LD_LIBRARY_PATH handling. Done means the CPU-based script installs and starts with the documented model defaults, while its GPU and API-key options are documented.