alibaba / alibaba/EasyNLP

SpanProto evaluation from checkpoint doesn't reproduce the f1 score

Open
#306 3 comments 0 reactions 1 assignee Claimed by @wjn1996 View on GitHub
Dominant language
Python
Stars
2.2k
Forks
257
PR merge metrics
No merged PRs in 30d

Description

Hi Team SpanProto,

As described in the README file of span-proto directory, I trained with few-nerd 5way 5shot inter dataset. The prediction result shows ~0.82 f1 score as presented in the paper. However, when I am trying to use the trained checkpoint for prediction only i.e. adding --resume_from_checkpoint=./outputs/bert-base-uncased or --resume_from_checkpoint=./outputs/bert-base-uncased/checkpoint-7000 etc and remove --do_train argument, the result is bad like ~0.1 f1 score. Could you please tell me how can I reproduce the result of the test set using the trained checkpoint once the training is complete?

Thanks,
sayef

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.