generate.py: error: argument --encoder: invalid choice: 'gpt'
- Dominant language
- Python
- Stars
- 3.1k
- Forks
- 520
- PR merge metrics
- No merged PRs in 30d
Description
I try to use the **MixedCorpus+GptEncoder+LmTarget** model for generate some text like GPT2. And I followed the example scripts like
```
python3 scripts/generate.py --pretrained_model_path models/gpt_model.bin --vocab_path models/google_zh_vocab.txt
--input_path story_beginning.txt --output_path story_full.txt --config_path models/bert_base_config.json
--encoder gpt --target lm --seq_length 128
```
Then it raise an error `generate.py: error: argument --encoder: invalid choice: 'gpt'`
It seems there are some codes mismatch because there is no such encoder type called gpt in UER code.
In addition to this, there are also something wrong with the pretrain scripts for GPT and the release GPT model from **MixedCorpus+GptEncoder+LmTarget**.
Some features in the pretrain scripts like
`--embedding word_pos --remove_embedding_layernorm --layernorm_positioning pre`
need to be droped if you wanna finetune the **MixedCorpus+GptEncoder+LmTarget** model.
And you need add another args `--has_lmtarget_bias`
Maybe you guys make some other scripts in the pretrain stage.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.