baidu / baidu/DuReader

运行DuReader/paddle/paragraph_extraction.py生成的数据中, answer_span对应的答案与fake_answer不相同

Open
#52 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
306
PR merge metrics
No merged PRs in 30d

Description

运行paddle中的run.sh脚本,调用paragraph_extraction生成新的文件后,发现 answer_span对应的答案与fake_answer对应的答案不是一直相同的。原则上二者应该相同吧?

下面3个例子取自demo/devset/search.dev.json生成的新的文件。可以看出二者大多数情况相同,但也有一些情况不相同。

```
question id 181623 fake_answer 大众牛逼关键在于是个德国车企,中国人普遍崇拜德国。
question id 181623 span_answer 我就想知道路上那么多大众怎么破,这不是事实么

question id 181625 fake_answer 结构不固定,形式多变,或正或反,或分或合,笔画或多或少,相当灵活,具有很大的随意性
question id 181625 span_answer

question id 181611 fake_answer 1、将干海参用自来水直接冲洗1分钟,洗掉表面少许微尘。2、置于1-10度凉纯净水中24小时左右,中间换水2次直至将海参泡软。3、将泡软的海参从腹部纵向剖开,去掉海参前端牙状物和体内白筋。4、添纯净水上无油锅加盖煮沸,用中火煮15-25分钟。5、换新的凉纯净水,泡24小时左右,中间换水2次直至发泡到2倍左右长度。6、泡好后,即可食用。可把多余的单独零度以下冷冻,建议2周内用完。7、如有个别海参没有发大,属于正常现象,可重复4、5步骤
question id 181611 span_answer 凉水泡24小时直至海参变软。第二步清洁剪掉海参的沙嘴,切断筋,清洗干净。第三步;将海参放入无油的,装入凉水的干净锅内,大火煮开改用小火煮50至60分钟左右,将海参捞出,用海参掐海参侧壁肉,能掐透或者稍变软即可,如没有则继续煮。第四步;水发,将煮好的海参捞出来,自然凉透之后
```

上面的结果是使用下面代码打印的。
```
with io.open(dev_path, 'r', encoding='utf-8') as fin:
data_set = []
for lidx, line in enumerate(fin):
sample = json.loads(line.strip())
if len(sample['answer_spans']) == 0:
continue
if len(sample['answer_docs']) == 0:
continue
if sample['answer_docs'][0] >= len(sample['documents']):
continue

print('fake answer', sample['fake_answers'][0])

answer_doc_idx = sample['answer_passages'][0]
start = sample['answer_spans'][0][0]
end = sample['answer_spans'][0][1]
print('span answer',
''.join(sample['documents'][answer_doc_idx]['segmented_paragraphs'][0][start: end + 1]))
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the examples by running paddle/run.sh and paragraph_extraction.py on demo/devset/search.dev.json. Inspect how fake_answers, answer_spans, answer_passages, and segmented_paragraphs are produced and printed, then determine why the selected text differs. Done means the reported mismatch is explained and the generated answer fields follow the intended consistency.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.