[RPaaS LiveValidation] Identify spike of heap memory usage in `validate-v2` image
- Dominant language
- C#
- Stars
- 135
- Forks
- 260
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 143
Description
We saw the `node` process running within the `validate-v2` image _crash_ with error:
> FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
Then, because the docker doesn't automatically restart the node process, `liveness` and `readiness` k8 probes fail, and the pod is restarted.
We have temporarily addressed this by raising the node heap size to 2GB. This didn't necessitate an upgrade in actual memory allocation to each k8 pod, so it was a painless update.
However, the RPaaS team is a bit leery (rightfully so) of just expanding memory and ignoring this problem. We want to understand _why_ this is occurring so that we can have a mitigation in place BEFORE the world explodes.
Contributor guide
Assessment
This issue has not been assessed yet.