confluentinc / confluentinc/ksql

Suggestion - Add Pagination to Table Queries

Open
#528 1 comment 3 reactions 0 assignees View on GitHub
enhancement P1 query-engine
Dominant language
Java
Stars
315
Forks
1k
Avg merge
1d 7h
Merged PRs (30d)
36

Description

The KSQL interface over REST currently returns an HTTP stream of data for a query on a table. This makes it difficult to serve multiple consumers (e.g. same table, multiple dashboards) and page through large KTables. This makes browsing data difficult for large amounts of data. A couple of suggestions:
* Add the ability to query a page of data using the `LIMIT` keyword, but also adding an `OFFSET` as the starting marker (e.g. the KTable key)
* Add the ability to consume KTable (and stream) updates over HTTP using a unique consumer id (similar to the Kafka REST Proxy).

Contributor guide

Open the contributing guide

Research direction

Start by locating the KSQL REST query interface and reviewing the Kafka REST Proxy behavior referenced in the issue. The request contains two separate proposals—page-oriented table queries and identified consumers for updates—so first clarify the intended scope and design. Done requires an agreed API behavior that supports the selected use case.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka, sql
Domain
api, databases, stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.