antirez / antirez/ds4

Request: Split large models into chunks

Abierto
#188 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

Downloading 500 GB is difficult, with mirrors, errors; have to download whole thing over again.

1. split model Delta, Delta, Delta, KV, FN/experts
- split along this

2. split every Delta Delta Delta KV into a chunk
- it speeds up loading with minio/s3 and NAS/NFS/cache
- make sure to split after delta delta delta kv fn, etc and dont split weird
- small files are easier for mirrors / and caching
- even small 40 GB files have faster load because parallel copy from nfs/s3/minio

3. I recommend naming your models like this

================================================================================
1A. ASCII DIAGRAM — THE 6-SECTION FORMAT
================================================================================

Template (field separator is a single literal "." between sections):

Section 1 Section 2 Section 3 Section 4 Section 5 Section 6
--------- --------- --------- --------- --------- ---------
. . . . . v
| | | | | |
v v v v v v
what who precision file publish re-derivation
model quantized container date counter
it
(dashes OK) (no dot) (dashes OK) (lowercase) (YYMMDD) ("v" + int)

Worked example — DeepSeek-V4-Flash IQ2XXS chat-v2 (no imatrix):

DeepSeek-V4-Flash . antirez . IQ2XXS-chat-v2 . gguf . 260518 . v1
|_________________| |_______| |______________| |____| |______| |__|
Section 1 Section 2 Section 3 Sec. 4 Section 5 S.6
Model Source Quant Format Date Ver

- leading capital on Section 1 (Model)
- Section 2 lowercase (HF org / publisher slug)
- Section 3 carries the publisher quant suffix ("chat-v2") since it is
part of the artifact's quantization identity, not the model identity
- Section 5 = 2026-05-18 collapsed to YYMMDD = 260518
- Section 6 starts at v1; bump on re-quant or re-derivation, NOT on
shard count, NOT on re-download of the same artifact

+----+------------+-----------------------------------------------+
| # | Section | Token shape |
+----+------------+-----------------------------------------------+
| 1 | Model | LeadingCapital, dashes-allowed, no dots |
| | | ex: DeepSeek-V4-Flash, Qwen3-Coder-Next |
| 2 | Source | lowercase slug, no dots |
| | | ex: antirez, unsloth, mlx-community, Intel |
| 3 | Quant | Mixed-case quant string, dashes for compounds |
| | | ex: IQ2XXS, Q4_K_M, UD-Q8_K_XL, FP8-Dynamic |
| 4 | Format | lowercase container ext |
| | | ex: gguf, safetensors, mlx, awq |
| 5 | Date | exactly 6 digits, YYMMDD |
| | | ex: 260518 means 2026-05-18 |
| 6 | Version | literal "v" + positive integer |
| | | ex: v1, v2, v10 |
+----+------------+-----------------------------------------------+

Upstream filename (verbatim from HF):
DeepSeek-V4-Flash-MTP-Q4K-Q8_0-F32.gguf

Normalized folder name:
DeepSeek-V4-Flash-MTP.antirez.Q4K-Q8_0-F32.gguf.260518.v1

---

This naming format, makes it much easier to track versions, modifications, etc

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.