opensearch-project / opensearch-project/OpenSearch
Support copying of files that are not associated with any ReplicationCheckpoint out to replicas.
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 13.7k
- Forks
- 3k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 108
Description
To support https://github.com/opensearch-project/OpenSearch/issues/10872 we need to be able to copy out merged segments to replicas outside of a refresh/commit cycle. The merged segments will be copied before they are part of any commit.
This means the files are not associated with a ReplicationCheckpoint, and existing segrep APIs (both node-node and remote store) operate solely on these checkpoints.
We need to design internal APIs that handle copying files that are outside of a checkpoint. This should support implementations for both node-node and remote store based copy.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing segrep APIs and their ReplicationCheckpoint assumptions for both node-node and remote-store copying. Done means defining internal APIs that can copy merged segments before they belong to any checkpoint, with implementations supported for both copy paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100