opensearch-project / opensearch-project/sql
[BUG] SQL Join timeout isn't applied when using PIT
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 176
- Forks
- 229
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 43
Description
What is the bug?
When using a JOIN timeout hint with a PIT-enabled cluster, the PIT will get deleted before the timeout expires.
Caused by: java.lang.IllegalArgumentException: Point In Time id: [y_T7MwDnSG-rblpw9Xa6mQ] doesn't exist
This is due to a 1-minute keepalive https://github.com/opensearch-project/sql/blob/35518dc185253e78f48b48184c37954ace576e63/opensearch/src/main/java/org/opensearch/sql/opensearch/setting/OpenSearchSettings.java#L61
How can one reproduce the bug?
Steps to reproduce the behavior:
- Create indices that require more than 60s to join on (>5 mil records each)
- Configure the cluster to use PIT joins per https://github.com/opensearch-project/sql/blob/35518dc185253e78f48b48184c37954ace576e63/docs/dev/opensearch-pagination.md?plain=1#L497
- Run a join query with a longer join timeout hint
/*! JOIN_TIME_OUT(90) */ - See in the logs that the join failed because the PIT no longer exists (deleted by the 1m keepalive)
What is the expected behavior?
The JOIN_TIME_OUT should also update the keepalive
What is your host/environment?
- OS: Latest
Do you have any screenshots?
N/A
Do you have any additional context?
Add any other context about the problem.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with opensearch/src/main/java/org/opensearch/sql/opensearch/setting/OpenSearchSettings.java and legacy/src/main/java/org/opensearch/sql/legacy/pit/PointInTimeHandlerImpl.java, then review the PIT configuration described in docs/dev/opensearch-pagination.md. Ensure a JOIN_TIME_OUT longer than the default keepalive keeps the PIT available through the join, and verify the reported timeout scenario no longer fails because the PIT was deleted.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100