confluentinc / confluentinc/ksql
Suggestion - Add Pagination to Table Queries
- Dominant language
- Java
- Stars
- 315
- Forks
- 1k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 36
Description
The KSQL interface over REST currently returns an HTTP stream of data for a query on a table. This makes it difficult to serve multiple consumers (e.g. same table, multiple dashboards) and page through large KTables. This makes browsing data difficult for large amounts of data. A couple of suggestions:
* Add the ability to query a page of data using the `LIMIT` keyword, but also adding an `OFFSET` as the starting marker (e.g. the KTable key)
* Add the ability to consume KTable (and stream) updates over HTTP using a unique consumer id (similar to the Kafka REST Proxy).
Contributor guide
Research direction
Start by locating the KSQL REST query interface and reviewing the Kafka REST Proxy behavior referenced in the issue. The request contains two separate proposals—page-oriented table queries and identified consumers for updates—so first clarify the intended scope and design. Done requires an agreed API behavior that supports the selected use case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka, sql
- Domain
- api, databases, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100