mlfoundations / mlfoundations/datacomp
Potential duplicates in rewritten subsets
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 792
- Forks
- 65
- PR merge metrics
- No merged PRs in 30d
Description
Hello DataComp team!
I'm seeking some clarification on the problem setup. To my understanding, when specifying a subset, if I assign a weight > 1 to a particular datapoint, it can appear multiple times in the rewritten dataset. This duplication may result in the same datapoint appearing twice in the same batch during contrastive training, potentially degrading performance (as the same datapoint would be contrasted against another copy of itself).
Do you have any mechanisms or suggestions within DataComp to help detect or handle these duplicate datapoints? If not, how would you recommend mitigating potential issues caused by having duplicates in the final dataset?
Thank you in advance for your guidance!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with resharder.py around line 948, where the issue says subset weights can duplicate datapoints in the rewritten dataset. Trace how weighted records flow into dataset batches and review the contrastive-training assumptions described in the issue. The scope is unresolved: completion would require an agreed mechanism or recommendation for detecting or handling duplicates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100