opensearch-project / opensearch-project/OpenSearch
Revamp peer recovery flow for remote backed indexes
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 13.7k
- Forks
- 3k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 108
Description
-
Currently, peer recovery for remote backed indexes works in 2 modes.
-
Target node downloads latest segments from remote segment store and the incremental segments (that are created on source while target was downloading the segments) are copied node-to-node from source to target.
-
This bi-modal behavior may create confusion or even regression if we build a feature that assumes no data copy between the two nodes.
-
In this feature request, we want to restrict data source at one place for remote backed indexes:
remote segment store -
More details to be added
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the issue description and investigate the current peer recovery flow for remote backed indexes, including the remote segment store download and source-to-target incremental segment copy. Clarify the intended recovery behavior and identify the relevant implementation and tests. Done means remote backed indexes use the remote segment store as their sole data source.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100