Spanproto recall为1.0的情况
- Dominant language
- Python
- Stars
- 2.2k
- Forks
- 257
- PR merge metrics
- No merged PRs in 30d
Description
您好!
在运行您提供的脚本后,发现recall一直为1.0的情况,不知是否是下面这行代码的原因(这里将真实标签也拼进去,后面评测的时候泄露了)https://github.com/wjn1996/SpanProto/blob/f1e0acb8672f0bfcbb7c827c48b06b3e8ccb295a/models/span_proto.py#L588
此外,在将此处改成`query_all_spans = query_predict_spans`后,得到的结果和论文中相差较大,不知哪边出了问题。
FEW-NERD 5way-1shot: inter—— span_f1:0.5826 class_f1:0.4618 intra——span_f1:0.4606 class_f1:0.3548
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at models/span_proto.py line 588 and run the provided evaluation script on FEW-NERD 5way-1shot. Check whether the true labels are included in query_all_spans, then compare the resulting inter and intra span_f1/class_f1 values with the reported paper results; done means the evaluation behavior and discrepancy are explained.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100