antirez / antirez/ds4

Request: Split large models into chunks

Đang mở
#188 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
C
Star
22.3k
Fork
2.1k
Merge trung bình
1 ngày 3 giờ
Pull request đã merge (30 ngày)
4

Mô tả

Downloading 500 GB is difficult, with mirrors, errors; have to download whole thing over again.

1. split model Delta, Delta, Delta, KV, FN/experts
- split along this

2. split every Delta Delta Delta KV into a chunk
- it speeds up loading with minio/s3 and NAS/NFS/cache
- make sure to split after delta delta delta kv fn, etc and dont split weird
- small files are easier for mirrors / and caching
- even small 40 GB files have faster load because parallel copy from nfs/s3/minio

3. I recommend naming your models like this

================================================================================
1A. ASCII DIAGRAM — THE 6-SECTION FORMAT
================================================================================

Template (field separator is a single literal "." between sections):

Section 1 Section 2 Section 3 Section 4 Section 5 Section 6
--------- --------- --------- --------- --------- ---------
. . . . . v
| | | | | |
v v v v v v
what who precision file publish re-derivation
model quantized container date counter
it
(dashes OK) (no dot) (dashes OK) (lowercase) (YYMMDD) ("v" + int)

Worked example — DeepSeek-V4-Flash IQ2XXS chat-v2 (no imatrix):

DeepSeek-V4-Flash . antirez . IQ2XXS-chat-v2 . gguf . 260518 . v1
|_________________| |_______| |______________| |____| |______| |__|
Section 1 Section 2 Section 3 Sec. 4 Section 5 S.6
Model Source Quant Format Date Ver

- leading capital on Section 1 (Model)
- Section 2 lowercase (HF org / publisher slug)
- Section 3 carries the publisher quant suffix ("chat-v2") since it is
part of the artifact's quantization identity, not the model identity
- Section 5 = 2026-05-18 collapsed to YYMMDD = 260518
- Section 6 starts at v1; bump on re-quant or re-derivation, NOT on
shard count, NOT on re-download of the same artifact

+----+------------+-----------------------------------------------+
| # | Section | Token shape |
+----+------------+-----------------------------------------------+
| 1 | Model | LeadingCapital, dashes-allowed, no dots |
| | | ex: DeepSeek-V4-Flash, Qwen3-Coder-Next |
| 2 | Source | lowercase slug, no dots |
| | | ex: antirez, unsloth, mlx-community, Intel |
| 3 | Quant | Mixed-case quant string, dashes for compounds |
| | | ex: IQ2XXS, Q4_K_M, UD-Q8_K_XL, FP8-Dynamic |
| 4 | Format | lowercase container ext |
| | | ex: gguf, safetensors, mlx, awq |
| 5 | Date | exactly 6 digits, YYMMDD |
| | | ex: 260518 means 2026-05-18 |
| 6 | Version | literal "v" + positive integer |
| | | ex: v1, v2, v10 |
+----+------------+-----------------------------------------------+

Upstream filename (verbatim from HF):
DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf

Normalized folder name:
DeepSeek-V4-Flash-MTP.antirez.Q4K-Q8_0-F32.gguf.260518.v1

---

This naming format, makes it much easier to track versions, modifications, etc

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.