Unexpected behavior modifying non-linear repository versions

Author: cottsay (cottsay)

Redmine Issue: 6811, https://pulp.plan.io/issues/6811

If an add/remove operation on a repository results in no changes from the previous version, a new version is not created and the task is completed with no entries in the created_resources list. This is a great feature, and is an easy to understand behavior.

However, this same behavior can be triggered in another circumstance, one which I believe is far less intuitive and difficult or impossible to handle atomically. In my opinion, this behavior is also difficult to understand without knowledge of the implementation of add/remove operations in Pulp.

If an add/remove operation is attempted based on a previous repository version which will result in the same set of content as the "latest" version at the time of the operation, the same behavior is exhibited: a successful task with no created_resources - even though the change results in a different set of content from the version on which the operation is specifically based.

The information returned by the operation is then insufficient to differentiate why the operation was "successful but not successful." If I follow-up with a subsequent query for the latest version to compare with my intended contents, I'm racing with another operation creating a new "latest" version.

I can see arguments for and against this latter behavior - refusing to create a new version when one already exists with the desired state. On one hand, it avoids duplicate versions. On the other, it only happens when the matching state happens to be the latest version even though the operation is specifically targeting another base version, and it complicates or eliminates the ability to use the chain of versions as a sort of revision history. Even if I manage to find which version the operation found to be an identical end-state, that state cannot contain the context of which version I'm coming from because it is already based on another version (by design).

Despite these drawbacks, if the behavior is kept, the add/remove operation should be modified to express which already existing repository version it found to match the state that the operation would result in, eliminating the need for subsequent queries and therefore eliminating the opportunity for a race.

From #pulp on Freenode:

cottsay: Happy weekend, everyone. I found some strange behavior related to non-linear repository versioning. It's hard to articulate, so I'm having trouble searching through bugs to find if it is a known issue or not, so I'm hoping someone can provide some insight. The problem occurs when creating a new repository version based on a version other than the latest, which happens to have the same end-state as the latest version.
cottsay: As I understand it, the new version is created by starting with the latest version and adding or removing content to align it was the specified base version: https://github.com/pulp/pulpcore/blob/master/pulpcore/app/models/repository.py#L99-L103
cottsay: If the add/remove operations I'm attempting happen to "undo" those under-the-hood add/remove operations, the operation triggers a "no-op" and no new version is created: https://github.com/pulp/pulpcore/blob/master/pulpcore/app/models/repository.py#L778-L779
cottsay: I can work around the problem by either a) performing a partial updated to get a new "latest version", then finish the update in a second operation (this only works if there are multiple add/remove operations pending) OR b) add extra logic to my add/remove scripting to detect that the target state happens to be the same as the latest version (specifically) and just use that version instead of creating a new one.
ggainey: "if you didn't make a change then a new version isn't created" is considered a feature of Pulp3 - you only get someting new, if there's a change
cottsay: Thanks for the response. I've observed that, and find it desirable. The problem is that the version I'm trying to create IS different from the base version I'm targeting, it just HAPPENS to be the same as the latest version in the repository. If an unrelated operation changes the content in the latest version, the same operation that is triggering the unexpected "no-op" will suddenly succeed in creating a new version.
ggainey: so if you versions 1 - 2 - 3 - 4, and you're creating a new-version based on '2', then 'latest' version will be your new version. If that happens to exactly match 4, you won't get a 5 - but latest will still be what your code resulted in
ggainey: so if you say "make these changes to 2 and give me the resulting version", it wil be 4. If something makes changes to latest, there will be a 5 - but you will still have '4' in hand
cottsay: Conversation is helping me understand, thank you. I find it less than intuitive that I'd need to detect this circumstance specifically. The modify/add-remove operation basically just returns "no change," and doesn't give any indication that the latest version is what I want. If I then go try to fetch the latest version, this introduces a race.
cottsay: I would find the behavior much more expected if the add-remove operation would return the (not) new version that I should be using, even if it didn't create it at that moment.
ggainey: now that sounds like a very valid RFE - nothing worse than code that knows what you want, and won't give it to you!
ggainey: can you open an issue for us please?

Start with pulpcore/app/models/repository.py at the referenced lines 99-103 and 778-779, then trace how add/remove operations report created_resources for non-linear versions. Done means the operation clearly identifies the existing repository version matching the requested end state, avoiding a race with a later latest-version query.