apollographql / apollographql/apollo-server
TCP RST and 502 errors when pods are terminated
- Dominant language
- TypeScript
- Stars
- 14k
- Forks
- 2k
- Avg merge
- 5h 30m
- Merged PRs (30d)
- 1
Description
### Issue Description
We see 502 errors when an apollo-server pod is gracefully terminated in k8s. We host apollo-server in EKS and expose it using AWS ALBs.
AWS provides a [troubleshooting guide for 502s](https://repost.aws/knowledge-center/elb-alb-troubleshoot-502-errors) and we can see our issue falls under the criteria for "The load balancer received a TCP RST from the target when attempting to establish a connection" (see screenshot below).
We attempted to increase the `stopGracePeriodMillis` in `ApolloServerPluginDrainHttpServer` to be higher than the kubernetes `terminationGracePeriodSeconds` and the ALB Target Group `deregistration_delay` but did not see a change in behavior.
We also have set `httpServer.keepAliveTimeout` and `httpServer.headersTimeout` higher than our ALB Session Timeout.

### Link to Reproduction
https://repost.aws/knowledge-center/elb-alb-troubleshoot-502-errors
### Reproduction Steps
1. Terminate a pod running running apollo-server in EKS behind an AWS ALB
Contributor guide
Research direction
Start with ApolloServerPluginDrainHttpServer and reproduce pod termination in EKS behind an AWS ALB, using the linked AWS troubleshooting guide as context. Determine where the TCP RST and resulting 502 occur during shutdown; done means identifying a repository-level fix or documenting that the behavior depends on the deployment configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes, node.js, typescript
- Domain
- backend, cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100