Return CloseableIterator from QueryWebSocketClient
- Dominant language
- Java
- Stars
- 107
- Forks
- 29
- Avg merge
- 19h 46m
- Merged PRs (30d)
- 141
Description
### Description / Background
Follows on from:
- https://github.com/gchq/sleeper/issues/5879
When we run a query through a web socket, with SleeperClient or QueryWebSocketClient, currently this returns a `CompletableFuture>`. This means that all the rows returned have to be collected into a list.
In the linked issue, we created an initial iterator implementation and
For this issue we can update the method signature to use a CloseableIterator. We will also need to update the tests to use the iterator. A future issue will actually return the rows as they are received from the web socket:
- https://github.com/gchq/sleeper/issues/6463
### Acceptance Criteria
**When** a query made to the SleeperClient or QueryWebSocketClient via a web socket
**Then** the method returns an iterator returning the rows one at a time
**And** when the iterator is closed, the web socket connection is closed
**And** for now we can still wait to receive all the rows before returning through the iterator, as this will change in a separate issue
### Technical Notes / Implementation Details
QueryWebSocketIterator implements CloseableIterator. We also prepared QueryWebSocketListener so we can wire the iterator into it gradually, replacing QueryWebSocketFuture.
We can:
1. Write a new submitQuery method alongside the existing one, that wires in QueryWebSocketIterator instead of QueryWebSocketFuture, updating QueryWebSocketClientTest to use the new method
2. Make new methods in SleeperClient to use the new method, updating the relevant tests in SleeperClientTest to use the new methods in SleeperClient
3. Remove the old SleeperClient methods
4. Remove the old QueryWebSocketClient.submitQuery method
5. Remove the QueryWebSocketFuture
6. Remove the QueryWebSocketHandler interface that let us have both QueryWebSocketFuture and QueryWebSocketIterator
Locations to be changed
- SleeperClient.queryViaWebSocket
- QueryWebSocketClient.submitQuery
- QueryWebSocketClientTest
- SleeperClientTest web socket tests
Contributor guide
Research direction
Start with SleeperClient.queryViaWebSocket and QueryWebSocketClient.submitQuery, then read QueryWebSocketClientTest and the web socket tests in SleeperClientTest. Update the client methods and tests to use the CloseableIterator, remove the old future and handler APIs listed in the issue, and verify that closing the iterator closes the web socket.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- api, backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100