AI-Hypercomputer / AI-Hypercomputer/maxtext

Gemma3 to hf mapping error

未關閉
#2,867 5 則留言 0 個 reaction 已指派 1 人 已被 @hengtaoguo 認領 在 GitHub 檢視
bug
主要語言
Python
星號
2.4k
分支
607
平均合併
2 天 19 小時
30 天內合併 PR
158

描述

### Bug report

After finetuning the model, I used the following command to convert my MaxText Gemma3 Checkpoint to HuggingFace format:

```bash
python3 -m MaxText.utils.ckpt_conversion.to_huggingface src/MaxText/configs/base.yml \ model_name='gemma3-4b' \
hf_access_token={hf token} \
load_parameters_path={my checkpoint path} \
base_output_directory=/tmp/gemma3-4B-cpt-hf \
use_multimodal=false # tried both false & true \
scan_layers=false # cannot create scanned checkpoint (another bug)
```

For testing purposes, I set up a vllm server on the TPU

```bash
pip3.11 install vllm-tpu

vllm serve /tmp/gemma3-4B-cpt-hf \
--disable-log-requests \
--tensor_parallel_size=4 \
--api-key {my api key}
```

Error message from vllm
```
ValueError: Gemma3 uses `gelu_pytorch_tanh` as the hidden activation function. Please set `hidden_act` and `hidden_activation` to `gelu_pytorch_tanh`.
```

[MaxText Conversion config](https://github.com/AI-Hypercomputer/maxtext/blob/c2574abeaad6e481d1ee62f961080f86b662a2dc/src/MaxText/utils/ckpt_conversion/utils/hf_model_configs.py#L38C31-L38C35) uses `"hidden_activation": "gelu"` for text generation

[huggingface config for text generation](https://huggingface.co/docs/transformers/v4.55.4/model_doc/gemma3#transformers.Gemma3TextConfig.hidden_activation) uses `gelu_pytorch_tanh`

[vllm](https://github.com/vllm-project/vllm/blob/c02a2705f9ceeb00b5d32453621f997b2ceafbea/vllm/model_executor/models/gemma3.py#L90) explicitly check for `gelu_pytorch_tanh` which resulted in the error above

### Logs/Output

### Environment Information

TPU creation command
```bash
export TPU_COUNT=4
gcloud alpha compute tpus tpu-vm create $TPU_ID \
--zone="${ZONE}" \
--accelerator-type="v6e-${TPU_COUNT}" \
--version=v2-alpha-tpuv6e \
--spot \
--service-account=${service_account}
```

Tried installation from source and uv as suggested in the documentation of MaxText.

### Additional Context

_No response_

貢獻指南

開啟貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。