pulp / pulp/pulpcore

Refactor Task's created_resources field to use an ArrayField or HStore

Open
#3,771 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Task
Dominant language
Python
Stars
598
Forks
168
Avg merge
1d 4h
Merged PRs (30d)
86

Description

From task query optimization PR: https://github.com/pulp/pulpcore/pull/3712:

created_resources currently uses a GenericReleationModel to store references to items created during a task. This can cause significant performance issues when quering tasks especially for relations that require Master-Detail related lookups. Also, when the associated resource gets deleted the task is left with a reference to null when serialized.

The logic around created_resources could be simplified if switched to a ArrayField or HStore, there would be no need for complicated lookup logic when quering tasks. Things to note for the conversion:

  • The CreatedResource model is heavily used in tasking code and in plugins, usually like so: CreatedResource(resource).save(),
  • created_resources is queriable through the REST API
  • Current created_resources that point to null are mostly recoverable if they were not pointing to a RepositoryVersion

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing CreatedResource(resource).save() through the tasking code and the created_resources REST API, then review task query optimization PR #3712. Done means converting the storage while preserving existing task references, plugin usage, and API querying, including handling resources that now serialize as null.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, database
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.