SpanProto evaluation from checkpoint doesn't reproduce the f1 score
- Dominant language
- Python
- Stars
- 2.2k
- Forks
- 257
- PR merge metrics
- No merged PRs in 30d
Description
Hi Team SpanProto,
As described in the README file of span-proto directory, I trained with few-nerd 5way 5shot inter dataset. The prediction result shows ~0.82 f1 score as presented in the paper. However, when I am trying to use the trained checkpoint for prediction only i.e. adding --resume_from_checkpoint=./outputs/bert-base-uncased or --resume_from_checkpoint=./outputs/bert-base-uncased/checkpoint-7000 etc and remove --do_train argument, the result is bad like ~0.1 f1 score. Could you please tell me how can I reproduce the result of the test set using the trained checkpoint once the training is complete?
Thanks,
sayef
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.