huggingface / huggingface/naacl_transfer_learning_tutorial

Label handling commit breaks the imdb finetuning script

オープン
#3 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
Python
スター
725
フォーク
118
PR マージ指標
30日以内にマージされた PR はありません

説明

Thomas, thanks for sharing this code! I noticed that commit 8d9c2371fc37ba8958f174501a9afb91a7ef7a06 seems to have broken the default functioning of the classification finetuning scripts - in the previous version there seems to have been a key called `'labels'` associated with the imdb and trec dictionaries, but in `finetuning_train.py` [this line](https://github.com/huggingface/naacl_transfer_learning_tutorial/blob/69031641946979f0ce1a0961a6b50e290f3de434/utils.py#L127) still references the now deleted key.

I updated the line to just use `DATASETS_LABELS_URL['imdb']['test']` as intended, but then it seems that the S3 bucket doesn't have the IMDB test file.

See below:
```
file_path = "https://s3.amazonaws.com/datasets.huggingface.co/imdb/test.labels.txt"
label_file = cached_path(file_path)
with open(label_file, "r", encoding="utf-8") as f:
all_lines = f.readlines()
print(all_lines[:5])
```
Gives:
```
['\n', 'NoSuchKeyThe specified key does not exist.imdb/test.labels.txt3D9E7C511167A0FBRiidOcrHfFaqxW9tmUXRppE/G3lsYoCZcq+uaYDi2yPPoe8mv/Og6PMuUncwk+B53tGsvcCZMWk=']
```

Does the test file for IMDB still exist with this name? This doesn't seem to be an issue with TREC.

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

Start in finetuning_train.py and the DATASETS_LABELS_URL definition referenced around utils.py line 127, then run the IMDB classification finetuning path. Check whether the configured S3 path for imdb/test.labels.txt matches an available test file and whether the default script completes without the deleted labels key. Done means IMDB and TREC label handling both work with the current dataset metadata.

索引モデルが issue の本文から書いたものです。

評価

技術スタック
aws, python
領域
data, machine-learning
issue の種類
バグ
難易度
3/5
見積もり時間
1〜2日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。