I installed it on Ubuntu Linux. Here some common help with that...

Here's a clear, structured summary you can copy-paste (or slightly adapt) into a GitHub issue, discussion, wiki page, or README section like “Running on Ubuntu 24.04 – Common Pitfalls & Fixes”.

Feel free to change the tone, add your username, screenshots or exact dates if you want.

markdown

Running Microsoft BitNet b1.58-2B-4T on Ubuntu 24.04 (Noble Numbat) – Experience & Fixes

March 2026 – tested on Ubuntu 24.04 LTS with fresh install + many PPAs already present.

Goal: run the official repo → https://github.com/microsoft/BitNet
Model: ggml-model-i2_s.gguf from Hugging Face microsoft/BitNet-b1.58-2B-4T-gguf

Summary of problems we hit & how we solved them

1. Installing recent Clang (needed ≥18, better ≥19–20)

sudo bash -c "$(wget -O - https://apt.llvm.org/llvm.sh)"
sudo apt update
sudo apt install clang-20 lld-20 cmake

→ installs clang-20 and clang++-20 (not plain clang / clang++)

2. CMake cannot find compiler → "clang is not a full path and was not found in the PATH"

Cause
setup_env.py hard-codes -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
→ looks for unversioned clang, but Ubuntu llvm.sh only creates versioned binaries.

Quick & clean workaround (worked best for us):

mkdir -p ~/bin
ln -s /usr/bin/clang-20   ~/bin/clang
ln -s /usr/bin/clang++-20  ~/bin/clang++

# Add to current session (or put in ~/.bashrc)
export PATH="$HOME/bin:$PATH"

After this which clang → /home/yourname/bin/clang and clang --version shows 20.x

3. Build fails in ggml-bitnet-mad.cpp with const-correctness error (Clang 20)

Error (line ~811):

cannot initialize a variable of type 'int8_t *' with an rvalue of type 'const int8_t *'
   int8_t * y_col = y + col * by;

Cause
Clang 20 is stricter about discarding const qualifiers than Clang 18 or GCC.

Fix (minimal patch – safe because pointer is only read from)

Edit file:

nano src/ggml-bitnet-mad.cpp

Change line ~811 from:

int8_t * y_col = y + col * by;

to:

const int8_t * y_col = y + col * by;

Save → re-run

python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s
# or directly:
cmake --build build --config Release -j$(nproc)

Alternative: use Clang 18 (less strict)

sudo apt install clang-18 lld-18
rm ~/bin/clang ~/bin/clang++
ln -s /usr/bin/clang-18 ~/bin/clang
ln -s /usr/bin/clang++-18 ~/bin/clang++
export PATH="$HOME/bin:$PATH"
rm -rf build
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s

4. Final working flow (after fixes)

# inside venv, in BitNet repo root
export PATH="$HOME/bin:$PATH"   # if not in .bashrc yet

# Download model if not already there
huggingface-cli download microsoft/BitNet-b1.58-2B-4T-gguf \
  ggml-model-i2_s.gguf --local-dir models/BitNet-b1.58-2B-4T

# Build (with the symlink trick + patch applied)
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s

# Run chat
python run_inference.py \
  -m models/BitNet-b1.58-2B-4T/ggml-model-i2_s.gguf \
  -p "You are a helpful assistant" \
  -cnv

Tips / Notes

• Model uses ~450–600 MB RAM → excellent for laptops / low-power machines
• Pure CPU right now (GPU support for i2_s quantization is still experimental in March 2026)
• If you want to try GPU later → build recent llama.cpp with -DLLAMA_CUDA=ON and test the same .gguf file with --n-gpu-layers 999
• Many warnings during compile (missing prototypes, unused params, double→float) are harmless

Hope this saves someone a few hours of frustration.

Tested on: Ubuntu 24.04, Ryzen CPU, Clang 20 from apt.llvm.org
Good luck & enjoy the tiny-but-surprisingly-useful model! 🚀

Feel free to add your hardware specs, approximate tokens/s, or any extra observations.

You can put this in:

- `docs/ubuntu-24.04-guide.md`
- a pinned Discussion thread
- or even as a new issue titled “Ubuntu 24.04 installation guide – common pitfalls & solutions”

Let me know if you want it shorter, more formal, or with extra sections (e.g. GPU attempt notes). Congrats again on getting it running!

Review the proposed Ubuntu 24.04 guide for docs/ubuntu-24.04-guide.md, then verify the referenced setup_env.py, src/ggml-bitnet-mad.cpp, and build commands on Ubuntu. Done means a focused guide accurately documents the Clang, CMake, const-correctness, model download, build, and inference steps.