hasura / hasura/graphql-engine

Streaming Subscriptions missing data when batch is smaller than amount of relevant rows

Open
#9,528 8 comments 0 reactions 0 assignees View on GitHub
k/enhancement t/docs
Dominant language
TypeScript
Stars
32.1k
Forks
3k
PR merge metrics
PR metrics pending

Description

### Version Information

Server Version: v2.21.0
CLI Version (for CLI related issue):

### Environment

OSS

### What is the current behaviour?

An entity have `updatedat timestamptz(3)` column, which is used to show when it was updated. The column is managed by before insert or update trigger. The test table contains two relevant rows.

The subscription:
```graphql
subscription organizationsChanges {
organizations_stream(cursor: {initial_value: {updatedat: "2000-01-01"}, ordering: ASC}, batch_size: 1) {
id
name
}
}
```
initially returns only one row, even though it should return two rows in the test case. Furthermore, if the rows are updated in the same transaction and therefore have the same timestamp, only one row is returned.

### What is the expected behaviour?

In my opinion, in this case the streaming subscription should return two batches of one row each.

### How to reproduce the issue?

1. Create a table with a timestamp or number and insert some rows.
2. Subscribe to the events with a batch smaller than the number of rows -> not all rows above the initial value will be returned.
3. Increase number or timestamp to all rows at once -> not all relevant rows are returned

### Screenshots or Screencast

-

### Please provide any traces or logs that could help here.

-

### Any possible solutions/workarounds you're aware of?

Increasing the batch size helps for some use cases, but not for software like task management and chat, which support mass data edits.

Another workaround is to stick with the non-streaming subscription.

### Keywords

stream streaming subscription batch cursor

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.