运行deepseek量化失败

1. 执行 python3 tools/run.py -c configs/deepseek_r1/fp8_static/deepseek_r1_fp8_static_low_memmory.yaml 出错

global:
  save_path: ./output-fp8

# Simplified Configuration for LLM compression
model:
  name: DeepSeek
  model_path: /workspace/models/DeepSeek-R1
  trust_remote_code: true
  low_cpu_mem_usage: true
  use_cache: false
  torch_dtype: fp8
  device_map: cpu

# Compression configuration
compression:
  name: PTQ
  quantization:
    name: fp8_static     # Supported: fp8_static, w4a8_fp8
    bits: 8                # Quantization bits (8)
    low_memory: True  # Use less gpu/cpu memory
    quant_method:
      weight: "per-tensor"
      activation: "per-tensor"
    ignore_layers:         # Skip quantization for these layers
      - "lm_head"
      - "model.embed_tokens"

# Dataset for calibration
dataset:
  name: TextDataset
  data_path: ./dataset/sharegpt_gpt4/sharegpt_gpt4_256.jsonl
  max_seq_length: 4096
  num_samples: 256
  batch_size: 1

在量化到最后

[2025-08-21 03:15:09] [AngelSlim] GPU Memory: 41642.00 MB
[2025-08-21 03:15:09] [AngelSlim] FP8 end layer 59

[2025-08-21 03:15:11] [AngelSlim] GPU Memory: 41642.00 MB
[2025-08-21 03:15:12] [AngelSlim] FP8 end layer 60

[2025-08-21 03:15:12] [AngelSlim] FP8 end batch 255

[2025-08-21 03:15:13] [AngelSlim] Start convert model...
Traceback (most recent call last):
  File "/workspace/AngelSlim/tools/run.py", line 238, in <module>
    run(config)
  File "/workspace/AngelSlim/tools/run.py", line 175, in run
    slim_engine.save(global_config.save_path, config)
  File "/workspace/AngelSlim/angelslim/engine.py", line 225, in save
    self.compressor.convert()
  File "/workspace/AngelSlim/angelslim/compressor/quant/ptq.py", line 136, in convert
    self._convert_llm()
  File "/workspace/AngelSlim/angelslim/compressor/quant/ptq.py", line 218, in _convert_llm
    qdq_module = self.quant_model.get_qdq_module(sub_layer, name)
  File "/workspace/AngelSlim/angelslim/models/llm/deepseek.py", line 166, in get_qdq_module
    group_size=self.quant_config.quant_algo_info["w_group_size"],
KeyError: 'w_group_size'

2. 执行 python3 tools/run.py -c configs/deepseek_r1/w4a8_fp8/deepseek_r1_w4a8_fp8_low_memmory.yaml 出错

# Global configuration of pipeline
global:
  save_path: ./output-w4a8
  deploy_backend: trtllm

# Simplified Configuration for LLM compression
model:
  name: DeepSeek
  model_path: /workspace/models/DeepSeek-R1
  trust_remote_code: true
  low_cpu_mem_usage: true
  use_cache: false
  torch_dtype: fp8
  device_map: cpu

# Compression configuration
compression:
  name: PTQ
  quantization:
    name: w4a8_fp8         # Supported: fp8_static, w4a8_fp8
    bits: 8                # Quantization bits (8)
    low_memory: True  # Use less gpu/cpu memory
    quant_method:
      weight: "per-group"
      group_size: 128
      activation: "per-tensor"
    ignore_layers:         # Skip quantization for these layers
      - "q_a_proj"
      - "q_b_proj"
      - "kv_a_proj_with_mqa"
      - "kv_b_proj"
      - "o_proj"
      - "mlp.gate_proj"
      - "mlp.up_proj"
      - "mlp.down_proj"
      - "shared_expert"

# Dataset for calibration
dataset:
  name: TextDataset
  data_path: ./dataset/sharegpt_gpt4/sharegpt_gpt4_256.jsonl
  max_seq_length: 4096
  num_samples: 128
  batch_size: 1

在量化到最后

[2025-08-20 21:45:45] [AngelSlim] FP8 end layer 60

[2025-08-20 21:45:45] [AngelSlim] FP8 end batch 127

[2025-08-20 21:45:46] [AngelSlim] Start convert model...
Traceback (most recent call last):
  File "/workspace/AngelSlim/tools/run.py", line 238, in <module>
    run(config)
  File "/workspace/AngelSlim/tools/run.py", line 175, in run
    slim_engine.save(global_config.save_path, config)
  File "/workspace/AngelSlim/angelslim/engine.py", line 225, in save
    self.compressor.convert()
  File "/workspace/AngelSlim/angelslim/compressor/quant/ptq.py", line 136, in convert
    self._convert_llm()
  File "/workspace/AngelSlim/angelslim/compressor/quant/ptq.py", line 186, in _convert_llm
    self.quant_model.act_scales_dict[name] = self.ptq_hook.observer_dict[
  File "/workspace/AngelSlim/angelslim/compressor/quant/observers/abs_max_activation.py", line 89, in scales
    raise ValueError(
ValueError: AbsmaxPertensorObserver scales must calibrate data first!

AbsmaxPertensorObserver 的错误似乎是因为校验数据较少就会导致？那么至少需要多少呢？