antirez / antirez/ds4

Request: Split large models into chunks

Aperta
#188 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
C
Stelle
22.3k
Fork
2.1k
Merge medio
1g 3h
PR unite (30g)
4

Descrizione

Downloading 500 GB is difficult, with mirrors, errors; have to download whole thing over again.

1. split model Delta, Delta, Delta, KV, FN/experts
- split along this

2. split every Delta Delta Delta KV into a chunk
- it speeds up loading with minio/s3 and NAS/NFS/cache
- make sure to split after delta delta delta kv fn, etc and dont split weird
- small files are easier for mirrors / and caching
- even small 40 GB files have faster load because parallel copy from nfs/s3/minio

3. I recommend naming your models like this

================================================================================
1A. ASCII DIAGRAM — THE 6-SECTION FORMAT
================================================================================

Template (field separator is a single literal "." between sections):

Section 1 Section 2 Section 3 Section 4 Section 5 Section 6
--------- --------- --------- --------- --------- ---------
. . . . . v
| | | | | |
v v v v v v
what who precision file publish re-derivation
model quantized container date counter
it
(dashes OK) (no dot) (dashes OK) (lowercase) (YYMMDD) ("v" + int)

Worked example — DeepSeek-V4-Flash IQ2XXS chat-v2 (no imatrix):

DeepSeek-V4-Flash . antirez . IQ2XXS-chat-v2 . gguf . 260518 . v1
|_________________| |_______| |______________| |____| |______| |__|
Section 1 Section 2 Section 3 Sec. 4 Section 5 S.6
Model Source Quant Format Date Ver

- leading capital on Section 1 (Model)
- Section 2 lowercase (HF org / publisher slug)
- Section 3 carries the publisher quant suffix ("chat-v2") since it is
part of the artifact's quantization identity, not the model identity
- Section 5 = 2026-05-18 collapsed to YYMMDD = 260518
- Section 6 starts at v1; bump on re-quant or re-derivation, NOT on
shard count, NOT on re-download of the same artifact

+----+------------+-----------------------------------------------+
| # | Section | Token shape |
+----+------------+-----------------------------------------------+
| 1 | Model | LeadingCapital, dashes-allowed, no dots |
| | | ex: DeepSeek-V4-Flash, Qwen3-Coder-Next |
| 2 | Source | lowercase slug, no dots |
| | | ex: antirez, unsloth, mlx-community, Intel |
| 3 | Quant | Mixed-case quant string, dashes for compounds |
| | | ex: IQ2XXS, Q4_K_M, UD-Q8_K_XL, FP8-Dynamic |
| 4 | Format | lowercase container ext |
| | | ex: gguf, safetensors, mlx, awq |
| 5 | Date | exactly 6 digits, YYMMDD |
| | | ex: 260518 means 2026-05-18 |
| 6 | Version | literal "v" + positive integer |
| | | ex: v1, v2, v10 |
+----+------------+-----------------------------------------------+

Upstream filename (verbatim from HF):
DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf

Normalized folder name:
DeepSeek-V4-Flash-MTP.antirez.Q4K-Q8_0-F32.gguf.260518.v1

---

This naming format, makes it much easier to track versions, modifications, etc

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.