huggingface / huggingface/xet-core

CAS Bridge not working for large files

Open
#592 8 comments 0 reactions 1 assignee Claimed by @rajatarya View on GitHub
bug
Dominant language
Rust
Stars
592
Forks
102
Avg merge
5d 8h
Merged PRs (30d)
9

Description

### Describe the bug

The Xet client allows larger files to be uploaded then can be downloaded over the CAS Bridge.

The increase of the per-file upload limit from 50 GB to 500 GB is highly appreciated as it finally allows us to upload GGUF quants of large models without splitting them heavily improving the user experience. It unfortunately turned out that such files can be only downloaded using the Xet Client but not over the HuggingFace website or any other download method that relays on the CAS Bridge. Please fix the CAS Bridge so it works with files up to the new 500 GB per-file limit.

The CAS Bridge simply errors with status code 400 when encountering such a file. This issue affects all recently created mradermacher quants of large models.

### Reproduction

HuggingFace Website downloads fails using `400 Bad Request`:
- https://huggingface.co/mradermacher/Mistral-Large-3-675B-Instruct-2512-i1-GGUF/blob/main/Mistral-Large-3-675B-Instruct-2512.i1-Q2_K.gguf
- https://huggingface.co/mradermacher/Mistral-Large-3-675B-Instruct-2512-i1-GGUF/blob/main/Mistral-Large-3-675B-Instruct-2512.i1-IQ3_M.gguf
- https://huggingface.co/mradermacher/Kimi-K2-Thinking-GGUF/blob/main/Kimi-K2-Thinking.Q6_K.gguf.part1of2

`git clone https://huggingface.co/mradermacher/Mistral-Large-3-675B-Instruct-2512-i1-GGUF` fails using `Smudge error: Error downloading Mistral-Large-3-675B-Instruct-2512.i1-IQ3_M.gguf`

XET client works: `hf download mradermacher/Mistral-Large-3-675B-Instruct-2512-i1-GGUF --include= Mistral-Large-3-675B-Instruct-2512.i1-Q2_K.gguf --local-dir=/GGUFs`

Files only slightly exceeding the old 50 GB limit work as intended:
- https://huggingface.co/mradermacher/shisa-v2.1-llama3.3-70b-i1-GGUF/blob/main/shisa-v2.1-llama3.3-70b.i1-Q6_K.gguf

### System info

Version used to upload the quants but should not matter:
```shell
- huggingface_hub version: 0.35.3
- Platform: Linux-6.17.2-2-pve-x86_64-with-glibc2.41
- Python version: 3.10.18
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Running in Google Colab Enterprise ?: No
- Token path ?: /root/.cache/huggingface/token
- Has saved token ?: True
- Who am I ?: mradermacher
- Configured git credential helpers:
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.6.0+cu124
- Jinja2: 3.1.6
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 11.3.0
- hf_transfer: N/A
- gradio: N/A
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: 2.11.7
- aiohttp: N/A
- hf_xet: 1.1.10
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /root/.cache/huggingface/hub
- HF_ASSETS_CACHE: /root/.cache/huggingface/assets
- HF_TOKEN_PATH: /root/.cache/huggingface/token
- HF_STORED_TOKENS_PATH: /root/.cache/huggingface/stored_tokens
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_DISABLE_XET: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.