NatLabRockies / NatLabRockies/COMPASS
Avoid duplicates in the database
@castelao is already working on this.
Since Jun 28, 2025.
- Dominant language
- Python
- Stars
- 17
- Forks
- 4
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 17
Description
The main concern here is on the trivial and clear cases of duplicates, such as running twice load in the same source. There are many 'levels' of duplication as well as many ways to recognize that. I'm interested here only on the trivial case of complete and exact duplication.
There is no good reason to load in the database twice the exact same data, and we should avoid that. The simplest approach would be by comparing the hash, but that would require a complete and precise hash, i.e. the hash must consider all the contents.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.