kvcache-ai / kvcache-ai/ktransformers
Minimax2.5 TypeError: non-default argument 'quantization_config' follows default argument
- Dominant language
- Python
- Stars
- 19.5k
- Forks
- 1.6k
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 27
Description
### Reminder
- [x] I have read the above rules and searched the existing issues.
### System Info
运行出错。
python -m sglang.launch_server \
--host 0.0.0.0 \
--port 30000 \
--model /mnt/nvme2/models/MiniMax-M2.5 \
--kt-weight-path /mnt/nvme2/models/MiniMax-M2.5 \
--kt-cpuinfer 25 \
--kt-threadpool-count 2 \
--kt-num-gpu-experts 36 \
--kt-method FP8 \
--kt-gpu-prefill-token-threshold 500 \
--kt-enable-dynamic-expert-update \
--attention-backend flashinfer \
--trust-remote-code \
--mem-fraction-static 0.85 \
--chunked-prefill-size 4096 \
--max-running-requests 32 \
--max-total-tokens 4096 \
--watchdog-timeout 3000 \
--enable-mixed-chunk \
--tensor-parallel-size 2 \
--enable-p2p-check \
--disable-shared-experts-fusion \
--fp8-gemm-backend triton \
--tool-call-parser minimax-m2 \
--log-level info \
--log-level-http info \
--uvicorn-access-log-exclude-prefixes /metrics \
--enable-metrics
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/xxx/miniconda/envs/kt2/lib/python3.11/site-packages/sglang/launch_server.py", line 30, in
from sglang.srt.server_args import prepare_server_args
File "/home/xxx/miniconda/envs/kt2/lib/python3.11/site-packages/sglang/srt/server_args.py", line 71, in
from sglang.srt.utils.hf_transformers_utils import check_gguf_file, get_config
File "/home/xxx/miniconda/envs/kt2/lib/python3.11/site-packages/sglang/srt/utils/hf_transformers_utils.py", line 78, in
from sglang.srt.configs.deepseek_v4 import DeepSeekV4Config
File "/home/xxx/miniconda/envs/kt2/lib/python3.11/site-packages/sglang/srt/configs/deepseek_v4.py", line 7, in
@dataclass
^^^^^^^^^
File "/home/xxx/miniconda/envs/kt2/lib/python3.11/dataclasses.py", line 1232, in dataclass
return wrap(cls)
^^^^^^^^^
File "/home/xxx/miniconda/envs/kt2/lib/python3.11/dataclasses.py", line 1222, in wrap
return _process_class(cls, init, repr, eq, order, unsafe_hash,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxx/miniconda/envs/kt2/lib/python3.11/dataclasses.py", line 1027, in _process_class
_init_fn(all_init_fields,
File "/home/xxx/miniconda/envs/kt2/lib/python3.11/dataclasses.py", line 545, in _init_fn
raise TypeError(f'non-default argument {f.name!r} '
TypeError: non-default argument 'quantization_config' follows default argument
deepseek_v4.py用ai辅助修改成下面内容后,这个错误消失。
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional
from transformers import PretrainedConfig
@dataclass
class DeepSeekV4Config(PretrainedConfig):
model_type = "deepseek_v4"
architectures: List[str]
quantization_config: Optional[Dict[str, Any]] = None
attention_bias: bool = False
attention_dropout: float = 0.0
bos_token_id: int = 0
eos_token_id: int = 1
ep_size: int = 1
first_k_dense_replace: int = 0
hidden_act: str = "silu"
hidden_size: int = 4096
index_head_dim: int = 128
index_n_heads: int = 64
index_topk: int = 512
initializer_range: float = 0.02
intermediate_size: int = 2048
kv_lora_rank: int = 512
max_position_embeddings: int = 65536
moe_intermediate_size: int = 2048
moe_layer_freq: int = 1
n_group: int = 8
n_routed_experts: int = 256
n_shared_experts: int = 1
norm_topk_prob: bool = True
num_attention_heads: int = 64
num_experts_per_tok: int = 6
num_hidden_layers: int = 43
num_key_value_heads: int = 1
q_lora_rank: int = 1024
qk_nope_head_dim: int = 448
qk_rope_head_dim: int = 64
rms_norm_eps: float = 1e-6
rope_scaling: Optional[Dict[str, float]] = None
rope_theta: int = 10000
routed_scaling_factor: float = 1.5
scoring_func: str = "sqrtsoftplus"
tie_word_embeddings: bool = False
topk_group: int = 8
topk_method: str = "noaux_tc"
use_cache: bool = True
v_head_dim: int = 512
vocab_size: int = 129280
o_lora_rank: int = 1024
o_groups: int = 8
window_size: int = 128
compress_rope_theta: int = 40000
compress_ratios: Optional[List[int]] = None
n_hash_layers: int = 3
hc_mult: int = 4
hc_sinkhorn_iters: int = 20
hc_eps: float = 1e-6
### Reproduction
```text
Put your message here.
```
### Others
_No response_
Contributor guide
Research direction
Start with sglang.srt.configs.deepseek_v4.py and the traceback from the supplied python -m sglang.launch_server command. Check the DeepSeekV4Config dataclass field ordering, then rerun the launch command with the MiniMax-M2.5 configuration; done means importing the server no longer raises the quantization_config TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai-infra-agents, backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100