antirez / antirez/ds4

deepseek tui support issue

Abierto
#554 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
C
Estrellas
22.3k
Forks
2.1k
Merge medio
1 d 3 h
PR fusionados (30 d)
4

Descripción

please check ds4 server hang when get request from deepseek tui
```
(ds4) root@salab-hpedl380g11-02:/mnt/nvme3n1/wayne/ds4/ds4# ./ds4-server --ctx 1000000 --kv-disk-dir /mnt/nvme3n1/ds4-kv --kv-disk-space-mb 2621440
ds4: Linux cuda backend set oom_score_adj=1000
ds4: CUDA backend initialized on NVIDIA L40S (sm_89)
ds4: CUDA registered 80.76 GiB model mapping for device access
ds4: CUDA preparing model tensor mappings: 80.24 GiB
ds4: CUDA q8 fp16 cache limit reached; using q8 kernels (request=64.00 MiB cached=7.94 GiB limit=8.00 GiB)
ds4: CUDA startup model preparation covered 80.76 GiB of tensor spans in 0.033s
ds4: cuda backend initialized for graph diagnostics
0712 13:00:04 ds4-server: context buffers 21302.79 MiB (ctx=1000000, backend=cuda, prefill_chunk=4096, raw_kv_rows=4352, compressed_kv_rows=250002)
ds4: CUDA using managed KV cache for ctx=1000000 (kv cache 13.17 GiB, context buffers 20.80 GiB); this may degrade performance but is needed for very large contexts
0712 13:00:04 ds4-server: KV disk cache /mnt/nvme3n1/ds4-kv (budget=2621440 MiB, cross-quant=accept, min=512, cold_max=30000, continued=10000, trim=32, align=2048, hit_half_life=21600s)
0712 13:00:04 ds4-server: listening on http://127.0.0.1:8000
0712 13:01:30 ds4-server: chat ctx=0..17642:17642 TOOLS prompt start
0712 13:01:30 ds4-server: chat ctx=0..17642:17642 TOOLS prefill chunk 0/17642 (0.0%) chunk=0.00 t/s avg=0.00 t/s 0.001s

```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.