getsentry / getsentry/sentry

[unmerging] Hash getting stuck in locked state

Open
#60,626 0 comments 0 reactions 0 assignees View on GitHub
Bug
Dominant language
Python
Stars
44.8k
Forks
4.9k
Avg merge
22h 21m
Merged PRs (30d)
586

Description

A customer reported they were unable to unmerge a specific fingerprint, because they tried once, didn't see the result (perhaps this was before the "please wait up to 24 hours for changes to take effect" warning was there), and then every time they tried again, they got an error that unmerging was already in progress.

I checked the hash in the `GroupHash` table (`sentry_grouphash`), and indeed, its state was set to `1`, which is the "locked in migration" state. 24 hours later, nothing had changed, so it seems to have gotten stuck that way somehow. The unmerging has not, in fact, happened (the hash is still associated with the old group in `sentry_grouphash`), so it seems that things fell off somewhere between the hash getting locked here

https://github.com/getsentry/sentry/blob/33283e4283f47691068cd920596d46b2e28606a3/src/sentry/tasks/unmerge.py#L502

and it getting unlocked here

https://github.com/getsentry/sentry/blob/33283e4283f47691068cd920596d46b2e28606a3/src/sentry/tasks/unmerge.py#L529.

This is the first report I've heard of this, and we don't seem to have any prior GH issues on the matter, either. I also can't reproduce it, so for now I'm going to chalk it up to gremlins, and we can work with the customer to get this particular hash unstuck. I'm writing up this issue just in case it happens again, though, so we have some place to collect the data, at which point we can think about investigating it further.

(For posterity, in case we do look further into this, the hash in question is `6618c48ad845c4be8fc8635ea060fd7f`, from project `6239706`, and it's currently stuck in group `3994122231`.)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.