Ensure that pulp imports can run concurrently with orphan cleanup
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 598
- Forks
- 168
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 86
Description
Author: daviddavis (daviddavis)
Redmine Issue: 8960, https://pulp.plan.io/issues/8960
I've confirmed that when an import happens, it bumps the timestamp_of_interest for artifacts and content so that when the import gets to the repository version creation step, the artifacts/content are still there.
However, I am not so sure that there isn't a possibly a race condition with django-import-export that could occur between when it selects the record(s) and then updates them. In fact, given the async problems we've seen in the past, I think it's likely.
I know we fixed another error similar to this:
https://pulp.plan.io/issues/8633
And perhaps our fix for #8633 might save us here: the fix added code to retry the import if it experienced errors and perhaps that will also apply to this situation where an artifact/content goes missing before it's updated. We need to confirm this though.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the retry fix for issue #8633 and the import behavior discussed in pulpcore#1367. Reproduce an import running concurrently with orphan cleanup, focusing on whether records can disappear between selection and update. Done means confirming whether the existing retry handling prevents the race or identifying the required correction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- backend, database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100