alipay / alipay/VCSL

Some labels are duplicate

Open
#13 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
144
Forks
21
PR merge metrics
No merged PRs in 30d

Description

Thanks for your great work on VCSL dataset! When checking test pair in `split_meta_pairs.json`, we found about 1% of the labels are self-duplicate (e.g. `e45b9eec5ea54e00a2d6e6689cd5fe92-e45b9eec5ea54e00a2d6e6689cd5fe92`, `dd44c4be9fdc4c95bdf075e7e756294e-dd44c4be9fdc4c95bdf075e7e756294e`) or query-reference reverse duplicate (e.g. `dd44c4be9fdc4c95bdf075e7e756294e-0a25c04af29940e5b34676b6c1a7eca1`(label:`[[65, 1, 79, 156]]`) and `0a25c04af29940e5b34676b6c1a7eca1-dd44c4be9fdc4c95bdf075e7e756294e`(label:`[[1, 65, 156, 79]]`)). And labels of some query-reference reverse duplicate pairs are different (e.g. `dd44c4be9fdc4c95bdf075e7e756294e-ca334995c55c45af93dee776799f0433`(label:`[[52, 9, 82, 44], [68, 84, 82, 96], [9, 1, 16, 7], [46, 97, 49, 100], [47, 45, 49, 47]]`) and `ca334995c55c45af93dee776799f0433-dd44c4be9fdc4c95bdf075e7e756294e` (label:`[[9, 52, 44, 82], [1, 9, 7, 16], [97, 46, 100, 49]]`)). These duplicate pairs may cause inaccurate results. I would like to ask that are these duplicate pairs made by mistake or by designed, thank you!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.