gchq / gchq/sleeper

Return CloseableIterator from QueryWebSocketClient

Open
#6,499 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Java
Stars
107
Forks
29
Avg merge
19h 46m
Merged PRs (30d)
141

Description

### Description / Background

Follows on from:
- https://github.com/gchq/sleeper/issues/5879

When we run a query through a web socket, with SleeperClient or QueryWebSocketClient, currently this returns a `CompletableFuture>`. This means that all the rows returned have to be collected into a list.

In the linked issue, we created an initial iterator implementation and

For this issue we can update the method signature to use a CloseableIterator. We will also need to update the tests to use the iterator. A future issue will actually return the rows as they are received from the web socket:

- https://github.com/gchq/sleeper/issues/6463

### Acceptance Criteria

**When** a query made to the SleeperClient or QueryWebSocketClient via a web socket
**Then** the method returns an iterator returning the rows one at a time
**And** when the iterator is closed, the web socket connection is closed
**And** for now we can still wait to receive all the rows before returning through the iterator, as this will change in a separate issue

### Technical Notes / Implementation Details

QueryWebSocketIterator implements CloseableIterator. We also prepared QueryWebSocketListener so we can wire the iterator into it gradually, replacing QueryWebSocketFuture.

We can:
1. Write a new submitQuery method alongside the existing one, that wires in QueryWebSocketIterator instead of QueryWebSocketFuture, updating QueryWebSocketClientTest to use the new method
2. Make new methods in SleeperClient to use the new method, updating the relevant tests in SleeperClientTest to use the new methods in SleeperClient
3. Remove the old SleeperClient methods
4. Remove the old QueryWebSocketClient.submitQuery method
5. Remove the QueryWebSocketFuture
6. Remove the QueryWebSocketHandler interface that let us have both QueryWebSocketFuture and QueryWebSocketIterator

Locations to be changed
- SleeperClient.queryViaWebSocket
- QueryWebSocketClient.submitQuery
- QueryWebSocketClientTest
- SleeperClientTest web socket tests

Contributor guide

Open the contributing guide

Research direction

Start with SleeperClient.queryViaWebSocket and QueryWebSocketClient.submitQuery, then read QueryWebSocketClientTest and the web socket tests in SleeperClientTest. Update the client methods and tests to use the CloseableIterator, remove the old future and handler APIs listed in the issue, and verify that closing the iterator closes the web socket.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.