FlowFuse / FlowFuse/flowfuse

Improve Node-RED startup time in k8s deployments

Open
#1,865 0 comments 0 reactions 0 assignees View on GitHub
epic
Dominant language
JavaScript
Stars
400
Forks
89
Avg merge
1d 20h
Merged PRs (30d)
149

Description

### Description

When we start a Node-RED instance, the launcher grabs the list of npm modules the user has added to their instance and runs an npm install.

Depending on the number of modules required, and their specific requirements, that npm install can be CPU/memory intensive. When running in k8s, with a limited CPU allocation, that can cause the install to take longer than expected. In the worst case, it can require more memory than has been allocation and the install gets killed. The only solution at that stage is to remove modules from the palette (via Instance Settings) and try again. If those were modules being actively used by the flow, they flows will be stopped until the user logs into the editor and manually reinstalls the modules one at a time (which avoids the spike in CPU/Memory).

This Epic is to decide on a plan of action to either avoid entirely or mitigate this scenario.

Some basic facts:
- We to limit the CPU/Memory of the Node-RED instance in order to distinguish between the different Instance Types available.
- Users will typically install modules one at a time via the editor. That works fine. But when they next suspend/resume, we have to reinstall everything in one go - which causes the CPU/Memory spike.
- There isn't a direct correlation between how many modules they install and how much memory is required. Some modules are more resource-intensive to get installed - such as those that require a binary component to get built.

There are some short-term tasks for this item:

- [ ] Notify user of out-of-memory issue when trying to start.
We do not currently report that clearly - it has to be inferred from the information available in the log. If we can detect it and notify the user more clearly, they will be in a better position to resolve it themselves rather than raising a support request.

There is a longer-term option to change how we start Node-RED. The basic outline is as follows:

- Add a mode to `nr-launcher` that does the work to pull down instance configuration and run npm install, but then exit cleanly (phase one)
- Add a mode to `nr-launcher` to skip the above bit and go straight to starting Node-RED. (phase two)
- When deploying to k8s, we introduce an 'init container' to the deployment that runs, without CPU/Memory constraint, the first phase of `nr-launcher`.
- When the init container exits cleanly, the main contain runs nr-launcher (phase two) - with the InstanceType CPU/Memory constraints in place.

By decoupling the install phase from the run phase we can apply different limits as needed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.