antirez / antirez/ds4

Request: Split large models into chunks

Ouverte
#188 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
C
Étoiles
22.3k
Forks
2.1k
Merge moyen
1 j 3 h
PR mergées (30 j)
4

Description

Downloading 500 GB is difficult, with mirrors, errors; have to download whole thing over again.

1. split model Delta, Delta, Delta, KV, FN/experts
- split along this

2. split every Delta Delta Delta KV into a chunk
- it speeds up loading with minio/s3 and NAS/NFS/cache
- make sure to split after delta delta delta kv fn, etc and dont split weird
- small files are easier for mirrors / and caching
- even small 40 GB files have faster load because parallel copy from nfs/s3/minio

3. I recommend naming your models like this

================================================================================
1A. ASCII DIAGRAM — THE 6-SECTION FORMAT
================================================================================

Template (field separator is a single literal "." between sections):

Section 1 Section 2 Section 3 Section 4 Section 5 Section 6
--------- --------- --------- --------- --------- ---------
. . . . . v
| | | | | |
v v v v v v
what who precision file publish re-derivation
model quantized container date counter
it
(dashes OK) (no dot) (dashes OK) (lowercase) (YYMMDD) ("v" + int)

Worked example — DeepSeek-V4-Flash IQ2XXS chat-v2 (no imatrix):

DeepSeek-V4-Flash . antirez . IQ2XXS-chat-v2 . gguf . 260518 . v1
|_________________| |_______| |______________| |____| |______| |__|
Section 1 Section 2 Section 3 Sec. 4 Section 5 S.6
Model Source Quant Format Date Ver

- leading capital on Section 1 (Model)
- Section 2 lowercase (HF org / publisher slug)
- Section 3 carries the publisher quant suffix ("chat-v2") since it is
part of the artifact's quantization identity, not the model identity
- Section 5 = 2026-05-18 collapsed to YYMMDD = 260518
- Section 6 starts at v1; bump on re-quant or re-derivation, NOT on
shard count, NOT on re-download of the same artifact

+----+------------+-----------------------------------------------+
| # | Section | Token shape |
+----+------------+-----------------------------------------------+
| 1 | Model | LeadingCapital, dashes-allowed, no dots |
| | | ex: DeepSeek-V4-Flash, Qwen3-Coder-Next |
| 2 | Source | lowercase slug, no dots |
| | | ex: antirez, unsloth, mlx-community, Intel |
| 3 | Quant | Mixed-case quant string, dashes for compounds |
| | | ex: IQ2XXS, Q4_K_M, UD-Q8_K_XL, FP8-Dynamic |
| 4 | Format | lowercase container ext |
| | | ex: gguf, safetensors, mlx, awq |
| 5 | Date | exactly 6 digits, YYMMDD |
| | | ex: 260518 means 2026-05-18 |
| 6 | Version | literal "v" + positive integer |
| | | ex: v1, v2, v10 |
+----+------------+-----------------------------------------------+

Upstream filename (verbatim from HF):
DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf

Normalized folder name:
DeepSeek-V4-Flash-MTP.antirez.Q4K-Q8_0-F32.gguf.260518.v1

---

This naming format, makes it much easier to track versions, modifications, etc

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.