elastic / elastic/docs-content
[Feedback]: Incorporation of PR #144339 (Internal Reindex Process Changes in Elasticsearch 9.4.0) into Documentation
- Dominant language
- No language data
- Stars
- 47
- Forks
- 261
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 116
Description
Dear Elastic Docs Support Team,
I hope this message finds you well.
We are currently building and operating a self-hosted Elasticsearch cluster version 8.18. Recently, when using the Reindex API, we encountered an issue with data count discrepancies. After consulting with Elastic Support, it was determined that the Reindex results were inaccurate due to shard rebalancing processes occurring simultaneously.
We understand that this issue has been addressed in PR #144339, which involves internal changes to the Reindex process in Elasticsearch 9.4.0.
However, the Elasticsearch v8.18 Reindex API documentation does not specifically mention the data count discrepancy issue caused by shard rebalancing. We believe that earlier documentation prior to version 9.4 also lacks this information.
Therefore, if possible, we kindly request that a note regarding this issue be added to the documentation.
We believe this would provide valuable information to other users facing similar situations.
Thank you for your attention and consideration.
Best regards,
---
## Elastic Docs AI Scoping 🤖
Docs issue scope
### Summary
The issue asks for a documentation note about potential data count discrepancies in the Reindex API when concurrent shard rebalancing occurs during a scroll-based reindex operation (affects v8.18 and earlier, also 9.0–9.4). PR [elastic/elasticsearch#144339](https://github.com/elastic/elasticsearch/pull/144339) replaces scroll-based pagination with Point-In-Time (PIT) pagination in the internal Reindex implementation — but this change was introduced **behind the `REINDEX_PIT_SEARCH_ENABLED` feature flag** in 9.4.0 and became the default in 9.5+. All 20 changed files in the PR are Java source code; no documentation files were modified.
### Request accuracy
Partially accurate — the PR does address the root cause by switching to PIT, but it became the default only in 9.5+ (not 9.4.0 as the issue states), and the current 9.x documentation already documents the scroll-versus-PIT distinction without yet adding an explicit caveat about count discrepancies for the scroll path.
### Next action for author
The documentation note about scroll-based reindex count discrepancies belongs in `elastic/elasticsearch` (not this `docs-content` repo) — file or request the change there, targeting `docs/reference/elasticsearch/rest-apis/reindex-indices.md` for the 9.0–9.4 scroll block (and the 8.x branch separately).
### Impact: Medium
### Scope boundary
No pages in this `docs-content` repository need updating; migration guides and how-tos here do not cover internal Reindex scroll/PIT mechanics at the level needed.
### Recommended documentation targets
| Page | URL | Action | Impact | Confidence | Why this page? |
|------|-----|--------|--------|------------|----------------|
| Reindex indices examples | https://www.elastic.co/docs/reference/elasticsearch/rest-apis/reindex-indices (current 9.x) | Add section to existing page | Medium | High | Contains the existing `applies-to` 9.0–9.4 scroll block where a caveat about count discrepancies belongs |
| Reindex API docs (8.x branch) | https://www.elastic.co/guide/en/elasticsearch/reference/8.18/docs-reindex.html | Update existing page | Medium | High | 8.x docs use scroll exclusively; a caveat or note here would directly address the user's request |
### Recommendations
1. In `elastic/elasticsearch`, add a caution or note inside the `{ "stack": "ga 9.0-9.4" }` applies-to block in `docs/reference/elasticsearch/rest-apis/reindex-indices.md` explaining that scroll-based reindex may report inaccurate document counts if shard rebalancing occurs concurrently, and recommend upgrading to 9.5+ where PIT pagination eliminates this issue.
2. In the `8.x` branch of `elastic/elasticsearch`, add a similar note to the Reindex API reference page for users still on 8.18 (the version the requester is using).
### Notes
- The existing 9.x page already has an `applies-to` callout block distinguishing PIT (9.5+) from scroll (9.0–9.4); the recommended change adds a caveat sentence to the scroll block only — no new page or new section heading is needed.
- The `>non-issue` label on the PR indicates this was an implementation improvement (not a bug fix), but the scroll count-discrepancy behavior is still a real user-facing limitation worth documenting.
- This issue should be re-filed or cross-linked in `elastic/elasticsearch` for the actual code change to land.
> Generated by [Issue Scope Analyzer](https://github.com/elastic/docs-content/actions/runs/27936253787) for issue #7048 · 86 AIC · ⌖ 7.1 AIC · ⊞ 8.8K · [◷](https://github.com/search?q=repo%3Aelastic%2Fdocs-content+is%3Aissue+%22gh-aw-workflow-call-id%3A+elastic%2Fdocs-content%2Fgh-aw-docs-issue-scope%22&type=issues)
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading docs/reference/elasticsearch/rest-apis/reindex-indices.md in the elastic/elasticsearch repository and compare its 9.0–9.4 scroll block with the 8.18 Reindex API page. Confirm the documentation change belongs there rather than in docs-content; done means the concurrent shard-rebalancing count discrepancy is documented for the applicable versions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100