f1 and em score is less than official result by 0.3%
- Ngôn ngữ chính
- Python
- Star
- 1.5k
- Fork
- 667
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
I followed the exact instructions in the 'readme.md' file and started training my model with the following command:
` python -m basic.cli --mode train --noload --len_opt --cluster --batch_size 50`
After 18K steps I used the following command to test the model
`python squad/evaluate-v1.1.py $HOME/data/squad/dev-v1.1.json out/basic/00/answer/test-####.json`
and then I got f1=74.982, exact_match=64.90.
The scores for a single model in the original paper are em=68.0 and f1=77.3. And mine are 0.3 % point lower than those.
Because the codes are provided by the official group , the hyper parameters are exactly the same except the batch_size which won't affect the models' performance critically. The only reason I can think of is the different initial value.
Has anyone done the same work as I do ? Or can anyone provides other ideas?
Thanks a lot!!!!
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.