AI-Hypercomputer / AI-Hypercomputer/maxtext
Gemma3 to hf mapping error
- Langage dominant
- Python
- Étoiles
- 2.4k
- Forks
- 607
- Merge moyen
- 2 j 19 h
- PR mergées (30 j)
- 158
Description
### Bug report
After finetuning the model, I used the following command to convert my MaxText Gemma3 Checkpoint to HuggingFace format:
```bash
python3 -m MaxText.utils.ckpt_conversion.to_huggingface src/MaxText/configs/base.yml \ model_name='gemma3-4b' \
hf_access_token={hf token} \
load_parameters_path={my checkpoint path} \
base_output_directory=/tmp/gemma3-4B-cpt-hf \
use_multimodal=false # tried both false & true \
scan_layers=false # cannot create scanned checkpoint (another bug)
```
For testing purposes, I set up a vllm server on the TPU
```bash
pip3.11 install vllm-tpu
vllm serve /tmp/gemma3-4B-cpt-hf \
--disable-log-requests \
--tensor_parallel_size=4 \
--api-key {my api key}
```
Error message from vllm
```
ValueError: Gemma3 uses `gelu_pytorch_tanh` as the hidden activation function. Please set `hidden_act` and `hidden_activation` to `gelu_pytorch_tanh`.
```
[MaxText Conversion config](https://github.com/AI-Hypercomputer/maxtext/blob/c2574abeaad6e481d1ee62f961080f86b662a2dc/src/MaxText/utils/ckpt_conversion/utils/hf_model_configs.py#L38C31-L38C35) uses `"hidden_activation": "gelu"` for text generation
[huggingface config for text generation](https://huggingface.co/docs/transformers/v4.55.4/model_doc/gemma3#transformers.Gemma3TextConfig.hidden_activation) uses `gelu_pytorch_tanh`
[vllm](https://github.com/vllm-project/vllm/blob/c02a2705f9ceeb00b5d32453621f997b2ceafbea/vllm/model_executor/models/gemma3.py#L90) explicitly check for `gelu_pytorch_tanh` which resulted in the error above
### Logs/Output
### Environment Information
TPU creation command
```bash
export TPU_COUNT=4
gcloud alpha compute tpus tpu-vm create $TPU_ID \
--zone="${ZONE}" \
--accelerator-type="v6e-${TPU_COUNT}" \
--version=v2-alpha-tpuv6e \
--spot \
--service-account=${service_account}
```
Tried installation from source and uv as suggested in the documentation of MaxText.
### Additional Context
_No response_
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.