DAMO-NLP-SG / DAMO-NLP-SG/ContProto
Running Error for bash run_generate_pseudo.sh de conll0
- Dominant language
- Python
- Stars
- 8
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
I try to run this code, however it seems that generate_pseudo.py could not runn correctly, the error is about this part:
dataset = BERTNERDataset(self.args, json_path=json_path,
tokenizer=BertWordPieceTokenizer(vocab_path),
# tokenizer=BertWordPieceTokenizer(vocab_file=vocab_path),
max_length=self.args.bert_max_length,
pad_to_maxlen=False
)
# dataset = BERTNERDataset(self.args, json_path=json_path,
# tokenizer=Tokenizer.from_pretrained(self.args.bert_config_dir),
# max_length=self.args.bert_max_length,
# pad_to_maxlen=False
# )
I'd like to know which type of tokenizer did you use when you train the model based on xlmr-large, It seems that both Tokenizer and BertWordPieceTokenizer Could not run correctly. I also tried the XLMRobertaTokenizer from transformers but its output is not incompatible with the output required by the current code. The code needs the output with ids,type_ids and offsets.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.