pulp / pulp/pulpcore

Ensure that pulp imports can run concurrently with orphan cleanup

Open
#2,021 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Task
Dominant language
Python
Stars
598
Forks
168
Avg merge
1d 4h
Merged PRs (30d)
86

Description

Author: daviddavis (daviddavis)

Redmine Issue: 8960, https://pulp.plan.io/issues/8960


I've confirmed that when an import happens, it bumps the timestamp_of_interest for artifacts and content so that when the import gets to the repository version creation step, the artifacts/content are still there.

However, I am not so sure that there isn't a possibly a race condition with django-import-export that could occur between when it selects the record(s) and then updates them. In fact, given the async problems we've seen in the past, I think it's likely.

I know we fixed another error similar to this:

https://pulp.plan.io/issues/8633

And perhaps our fix for #8633 might save us here: the fix added code to retry the import if it experienced errors and perhaps that will also apply to this situation where an artifact/content goes missing before it's updated. We need to confirm this though.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the retry fix for issue #8633 and the import behavior discussed in pulpcore#1367. Reproduce an import running concurrently with orphan cleanup, focusing on whether records can disappear between selection and update. Done means confirming whether the existing retry handling prevents the race or identifying the required correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
backend, database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.