line / line/centraldogma

Consider ways to commit repo changes and metadata changes atomically

Open
#658 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
668
Forks
135
Avg merge
4d 19h
Merged PRs (30d)
7

Description

When repositories are updated, the corresponding metadata is updated afterwards sequentially.

https://github.com/line/centraldogma/blob/d596a340e68598afd981cfd8e3d18f34761f5eb9/server/src/main/java/com/linecorp/centraldogma/server/internal/api/RepositoryServiceV1.java#L125-L127

However, it is possible that operations on repository succeed, but operations on metadata fail.

This scenario has two consequences:

  1. The UI shows inconsistent states.
  2. Api calls will fail until the repository and metadata are synced to the same state.

Below is an example where removeRepo fails, but this scenario can happen with any of createRepo, restoreRepo

Sample case

Assume that repository removal succeeds, but the corresponding metadata removal fails

https://github.com/line/centraldogma/blob/d596a340e68598afd981cfd8e3d18f34761f5eb9/server/src/main/java/com/linecorp/centraldogma/server/internal/api/RepositoryServiceV1.java#L146

This means the following patch would've failed, and /repos/{repoName}/removal doesn't exist under dogma/metadata.json.

https://github.com/line/centraldogma/blob/d596a340e68598afd981cfd8e3d18f34761f5eb9/server/src/main/java/com/linecorp/centraldogma/server/metadata/MetadataService.java#L350-L355

From this state there are two problems..

  1. From the UI, it isn't possible to restore/purge the repository since the UI depends on dogma/metadata.json information. (The UI shows that the repository is still in the created state)
  2. For paths from the API, restoration is possible but an exception will be thrown when trying to update the metadata (RemoveOperation requires that the path exists). So the repository will be restored, but the rest API http response will indicate failure.

https://github.com/line/centraldogma/blob/d596a340e68598afd981cfd8e3d18f34761f5eb9/server/src/main/java/com/linecorp/centraldogma/server/metadata/MetadataService.java#L387-L390

I'm still unsure how to approach this issue, but leaving this issue for reference

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked update paths in server/src/main/java/com/linecorp/centraldogma/server/internal/api/RepositoryServiceV1.java and the corresponding operations in server/src/main/java/com/linecorp/centraldogma/server/metadata/MetadataService.java. Trace createRepo, restoreRepo, and removeRepo, focusing on failures between repository and metadata updates. Done means these operations cannot leave repository and metadata in divergent states, with the affected UI and API flows remaining consistent.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.