Reduce downtime when upgrading an instance
- Dominant language
- JavaScript
- Stars
- 400
- Forks
- 89
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 146
Description
### Description
Currently when an instance is upgraded to a new stack, we stop the existing instance, apply the update and restart the instance.
Part of the restart involves re-running `npm install` to ensure all dependencies are installed. This can take anything from seconds to minutes to complete - all the while, Node-RED isn't running.
The purpose of this epic is to capture the work needed to reduce this downtime. We will achieve this by doing all of the preparation of the updated instance *before* stopping the old instance. Once the updated instance is ready to run, we stop the old instance and start the new.
> [!note]
> This is likely to be limited to k8s as it depends on being able to run two instances in parallel with traffic routed to one before switching over.
This will require
- a flag on `nr-launcher` telling it to prepare the environment and Node-RED without starting the flows (currently k8s only)
- add onReady probe endpoint to nr-launcher for k8s to poll, returns read when npm install and NR startup complete but not flow start.
- ~an exposed api on `nr-launcher` to start the flows~
- updated driver to coordinate the old/new instances - with all of the necessary error handling to ensure we tidy things up properly.
- ~driver will also need to store state properly to ensure a forge app restart mid Node-RED restart will not leave things in limbo and orphaned Deployments~
- check that when editing a Deployment we can add onReady probes (e.g. the upgrade from old style to new)
This work can also lay the basis for an active/passive HA mode - where a second instance can be started as quickly as possible when the primary is unavailable. How we define 'unavailable' needs more research.
Contributor guide
Assessment
This issue has not been assessed yet.