dbiir / dbiir/UER-py

generate.py: error: argument --encoder: invalid choice: 'gpt'

Open
#110 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.1k
Forks
520
PR merge metrics
No merged PRs in 30d

Description

I try to use the **MixedCorpus+GptEncoder+LmTarget** model for generate some text like GPT2. And I followed the example scripts like
```
python3 scripts/generate.py --pretrained_model_path models/gpt_model.bin --vocab_path models/google_zh_vocab.txt
--input_path story_beginning.txt --output_path story_full.txt --config_path models/bert_base_config.json
--encoder gpt --target lm --seq_length 128
```
Then it raise an error `generate.py: error: argument --encoder: invalid choice: 'gpt'`
It seems there are some codes mismatch because there is no such encoder type called gpt in UER code.
In addition to this, there are also something wrong with the pretrain scripts for GPT and the release GPT model from **MixedCorpus+GptEncoder+LmTarget**.
Some features in the pretrain scripts like
`--embedding word_pos --remove_embedding_layernorm --layernorm_positioning pre`
need to be droped if you wanna finetune the **MixedCorpus+GptEncoder+LmTarget** model.
And you need add another args `--has_lmtarget_bias`
Maybe you guys make some other scripts in the pretrain stage.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.