apollographql / apollographql/graphql-subscriptions
using subscriptions for persistent messages
- Dominant language
- TypeScript
- Stars
- 1.6k
- Forks
- 129
- PR merge metrics
- No merged PRs in 30d
Description
has any thought been put into using graphql subscriptions for server-to-server communication usually done on queues? I’d really like to do something like that in our app, but the current model subscriptions use don’t seem to be robust enough for it. I made a subscription server that receives requests over http and responds over sns/sqs. It plays nice with the redis pubsub, for multi-server support. But it wouldn’t survive a server node bouncing. There are a number of ways to reconnect, but nothing that would guarantee data wasn’t lost. Fundamentally the problem seems to be the use of async iterators, which are essentially in-memory listeners. If a node is lost, there isn’t a good way to recover without possibly losing events. I could persist events immediately within the pubsub library to make sure they weren’t lost there, but what I really want is to be able to guarantee that processing the events is transactional, which listeners don’t really lend themselves to. I feel like I could solve this, but not within the current subscriptions model. Has this been discussed in the past?
I can solve the problem if I persist incoming queries (I'm using redis), and when publish calls are made, iterating over subscription resolvers to see which apply. It requires me to re-implement more of the subscription logic I would like, and more importantly, I had to alter the api for subscription resolvers adding two properties:
```
Subscription: {
strengthUpdated: {
events: ['strength'],
applies: (payload, variables) => variables.id === payload.id,
subscribe: withFilter(
(() => pubsub.asyncIterator(['strength'])),
(payload, variables) => variables.id === payload.id
),
resolve: (payload, args, context, query) => getCharacter(payload.character)
}
},
```
For my SNS/SQS transport, I don't use the subscribe property. But the websocket implementation still uses it. I could actually generate it in this case from the `event` and `applies` properties I added.
I don't really like deviating from your existing library this much, so I'm hoping you have some direction.
Contributor guide
Research direction
Start by reviewing the existing subscription model, async iterator behavior, and pubsub transports described in the issue, then compare them with the proposed SNS/SQS and Redis persistence approach. Done would require an agreed design for durable, transactional event processing and a defined implementation scope; the issue does not name specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, redis, typescript
- Domain
- backend-api-design, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100