hasura / hasura/graphql-engine

Subscriptions lock/deadlock write operations

Open
#9,706 9 comments 9 reactions 0 assignees View on GitHub
k/enhancement t/gql-services
Dominant language
TypeScript
Stars
32.1k
Forks
3k
PR merge metrics
PR metrics pending

Description

### Is your proposal related to a problem?

The problem: I'm not able to perform write operations on the same table that has active subscriptions on.

At the moment, the only way to configure live query subscriptions is `HASURA_GRAPHQL_LIVE_QUERIES_MULTIPLEXED_REFETCH_INTERVAL`. Which, as I understand, just set the interval between SQL queries to a table. Correct me if I am wrong.

Hypothesis:
```md
In my case, subscriptions execute so long that:
1. They fully load single core of PostgreSQL process
2. Have no actual interval, read SQL queries are executed continuously.
This results in complete lock of the table for write operations, so the only way to stop is to shut down web applications.
```

### Describe the solution you'd like

Write operations should be possible with long-lasting SQL queries for subscriptions (aka live queries).

### Describe alternatives you've considered

0. Write operations should be put between read operations, even if subscription result is loading longer than interval. Maybe we should change the meaning of the interval, so it is an interval between attempts to load the next version of data, so write operations can queue in that 1 second that is available by default.
1. It might not always possible to optimize queries, and it might be not easy to control whether subscriptions are active, any client from the internet can open a web page and activate subscriptions.
2. Or may be queries that execute too long, should be dropped by timeout or with error? So it would be easier to debug individual long executing subscriptions. Short subscription queries that execute fast should not be timed out only long ones, and it should be configurable.
3. There should be a way to update `HASURA_GRAPHQL_LIVE_QUERIES_MULTIPLEXED_REFETCH_INTERVAL` dynamically when Hasura is running if the meaning is unchanged. So no need to restart Hasura to change the interval. That at least will give an ability to adjust interval using Hasura API.
4. Hasura can automatically create materialized view for each subscription, so these are updated as fast/slow as it possible. But the subscription itself works on top of that materialized view. Anyway, between updates of materialized view it should be possible to query/inject some write operations, otherwise the system will be not usable.
5. Any other way of caching can be used to help mitigate such issues.
6. Or maybe we just clone the entire database and sync changes asynchronously, so queries are implemented on top of a copy that is not in real-time sync with the actual database. That will mean we can do regular queries and write operations just fine on the original database.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported behavior with PostgreSQL subscriptions and a write to the same table, while examining HASURA_GRAPHQL_LIVE_QUERIES_MULTIPLEXED_REFETCH_INTERVAL. Compare the proposed interval, timeout, and caching alternatives, then define a measurable result where writes remain possible during long-running subscriptions.

Written by the indexing model from the issue text.

Assessment

Tech stack
graphql, postgres
Domain
backend-api-design, databases, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.