opensearch-project / opensearch-project/OpenSearch
Fix non-zero reference count for replica shard files in CompositeDirectory
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 13.7k
- Forks
- 3k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 108
Description
Describe the bug
When a new file is created in CompositeDirectory, its reference count is increased by 1 in the FileCache. This count is normally decreased when the file is uploaded to remote storage. However, for replica shards, a segment_n file is created but never uploaded, leaving its reference count at 1. This causes problems during index deletion, as some files can't be deleted due to non-zero references.
We need to evaluate two options:
- Prevent file creation on replica shards for CompositeDirectory.
- If files must be created on replica shards, ensure they don't have non-zero references in the FileCache.
This issue affects clean index deletion and may lead to resource leaks if not addressed.
Meta Issue: https://github.com/opensearch-project/OpenSearch/issues/13149
Related component
No response
To Reproduce
- Go to '...'
- Click on '....'
- Scroll down to '....'
- See error
Expected behavior
.
Additional Details
Plugins
Please list all plugins currently enabled.
Screenshots
If applicable, add screenshots to help explain your problem.
Host/Environment (please complete the following information):
- OS: [e.g. iOS]
- Version [e.g. 22]
Additional context
Add any other context about the problem here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating CompositeDirectory and FileCache, then trace how replica-shard segment_n files are created, uploaded, and released. Compare the two proposed handling options and verify the chosen behavior against index deletion; done means replica files no longer retain non-zero references and clean deletion succeeds, with regression coverage for the failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems, search
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100