about the model name
- 主要言語
- Python
- スター
- 2.2k
- フォーク
- 286
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
```python
model_name = 'pre-model/' + 'longformer-encdec-base-16384'
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
print(model)
print("***" * 48)
```
=================================================
OUTPUT:
BartModel(....)
===================================================
```python
batch_sentences = ["Hello I'm a single sentence",
"And another sentence",
"And the very very last one"]
batch_of_second_sentences = ["I'm a sentence that goes with the first sentence",
"And I should be encoded with the second sentence",
"And I go with the very last one"]
encoded_inputs = tokenizer(batch_sentences, batch_of_second_sentences)
print(encoded_inputs)
print("***" * 48)
```
==================================================
OUTPUT:
{'input_ids': [[0, 31414, 38, 437, 10, 881, 3645, 2, 2, 100, 437, 10, 3645, 14, 1411, 19, 5, 78, 3645, 2], [0, 2409, 277, 3645, 2, 2, 2409, 38, 197, 28, 45320, 19, 5, 200, 3645, 2], [0, 2409, 5, 182, 182, 94, 65, 2, 2, 2409, 38, 213, 19, 5, 182, 94, 65, 2]], 'attention_mask': [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1], [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}
====================================================
I use hugggingface to load your model,and then to do the sentence pairs.Firstly, I print the model and it displays "BartModel(....)", is it correct? Secondly I found "encoded_inputs" may lack token_type_ids, So can the longformer model deal with a pair of sentences?Thanks!
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
評価
この issue はまだ評価されていません。