Prevent duplicate commit insertion on facade re-processing
- Dominant language
- Python
- Stars
- 13
- Forks
- 17
- Avg merge
- 6h 59m
- Merged PRs (30d)
- 1
Description
> [!NOTE]
> Migrated from [augurlabs/augur#3790](https://github.com/augurlabs/augur/issues/3790)
> Originally opened by `@MoralCode` on 2026-03-25
---
the `commits` table has a primary key that is derived only from a sequence (i.e. an autoincrementing integer)
When facade `analyze_commits_in_parallel` runs (to insert all new commit change information into the commits table), it does not utilize Augur's existing upsert (`bulk_insert_dicts` or similar) logic. Instead it simply inserts always, causing new IDs to be generated and new rows to be added.
If the run of analyzing commits is a rerun (i.e. the repo previously was fully collected, but the admin reset the last collection date to force recollection, meaning many of the commits are already in the table), this will simply generate duplicate rows, contributing to the size growth of one of the largest tables in Augur.
In order to use upserts for the commits table, we need a compound primary key based on the actual data. Given this table is actually more accurately described as commit_changes (#211), i propose this constraint `UniqueConstraint("repo_id", "cmt_commit_hash", "cmt_filename", name="commit-changes-unique"),`.
Contributor guide
Research direction
Start with the facade's analyze_commits_in_parallel entry point and compare its insertion path with the existing bulk_insert_dicts upsert logic. Inspect the commits table schema and determine how the proposed repo_id, cmt_commit_hash, and cmt_filename uniqueness constraint fits the storage layer. Done means rerunning commit analysis does not create duplicate rows.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100