FlowFuse / FlowFuse/node-red-dashboard

Increasing server CPU usage by chart if incoming messages include additional properties

Open
#2,151 3 comments 0 reactions 0 assignees View on GitHub
needs-triage
Dominant language
HTML
Stars
355
Forks
82
Avg merge
4d 23h
Merged PRs (30d)
24

Description

### Current Behavior

If messages passed to the ui-chart node include significant amounts of additional data then the node red process uses an increasing amount of CPU time as the number of data points on the chart increases.
For the example flow below, running node-red on an Intel i7, initially node-red shows about 2% usage. The flow injects a message each second, and over about 20 mins the CPU slowly increases to around 8%. Stopping the incoming messages immediately drops the CPU usage back to 2%, and releasing them again switches it back to 8%. Obviously, using 6% of an i7 core on just buffering up messages is ridiculous.

It is not a browser related issue as the symptom can be seen even if there is no browser open.

The fact that it increases slowly means that it not to do with rolling the message buffer as it is not necessary for the buffer to be full.

The message is significant but not huge, this is the function generating it
```
msg = {
"payload": -433,
"topic": "M3 (Measured)",
"mac": "3C1ACCBAA19D",
"data_type": "int16",
"property": "battery_power",
"communication_status": "operational",
"scale": 1,
"unit": "W",
"precision": 1,
"friendly_name": "Battery Power",
"device_class": "power",
"state_class": "measurement",
"entry_type": "SENSOR_DEFINITIONS",
"_msgid": "16df34e86e841df2",
"messageId": "6a3d7abd40cb157a90acc984",
"modbusRequest": {
"unitid": 1,
"fc": 3,
"address": 30001,
"quantity": 1,
"emptyMsgOnFail": true,
"keepMsgProperties": true,
"enableDeformedMessages": false,
"messageId": "6a3d7abd40cb157a90acc984"
},
"responseBuffer": {
"data": [
65103
],
"buffer": [
254,
79
]
},
"parse_buffer": [
254,
79
],
"specification": {
"options": {
"byteSwap": false,
"resultType": "keyvalue",
"singleResult": true,
"msgProperty": "translated_payload",
"setTopic": true
},
"items": [
{
"name": "value",
"type": "int16be",
"offset": 0,
"length": 1,
"mask": "",
"scale": "1",
"offsetbit": 0,
"id": 0,
"value": -433
}
]
},
"originalPayload": [
65103
],
"values": [
-433
],
"objectResults": {
"value": {
"name": "value",
"type": "int16be",
"offset": 0,
"length": 1,
"mask": "",
"scale": "1",
"offsetbit": 0,
"id": 0,
"value": -433
}
},
"keyvalues": {
"value": -433
},
"arrayResults": [
{
"name": "value",
"type": "int16be",
"offset": 0,
"length": 1,
"mask": "",
"scale": "1",
"offsetbit": 0,
"id": 0,
"value": -433
}
],
"buffer": [
254,
79
],
"translated_payload": {
"value": -433
},
"_event": "node:53ebc095797a50fc"
}
msg.payload = Math.floor(Math.random() * (2000 - (-1000) + 1)) + (-1000);
return msg;
```

Adding a node in front of the chart which moves just the payload and topic into a new message and passing that to the chart stops the issue from occurring.

The problem was identified by a user in forum post https://discourse.nodered.org/t/feasability-of-migrating-to-dashboard-2-0/101298.

I am investigating to find which part of the code is using up the CPU.

### Expected Behavior

_No response_

### Steps To Reproduce

~~Run this flow and watch the node-red CPU usage over about 20 minutes.~~

This is a better flow for investigating the issue. It passes in 10 messages per second, which should still be no problem for the server. Running on an Intel i7 the node-red cpu usage climes to 50% in just a couple of minutes. The flow also includes Inject nodes to block or release the incoming messages and a function node (Reduce) which can be inserted to remove the additional data in the message.

```
[{"id":"c12532a704294d5c","type":"group","z":"371f5f719cb5e2ab","name":"Chart CPU test. Issue #2151","style":{"label":true},"nodes":["893419d21d52a0f3","392c75611b71aaa9","1a80648df0d6abfd","69c0edb80b6c835a","457d9fc3638fa678","331341489f068df6","0642db45d40d5b8f","e8772e058c096989","cc612ff604efa32f","909073dac6635927","33366f32a7a526c7"],"x":54,"y":79,"w":892,"h":222},{"id":"893419d21d52a0f3","type":"inject","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"1","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":170,"y":220,"wires":[["392c75611b71aaa9"]]},{"id":"392c75611b71aaa9","type":"function","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"Create object","func":"msg = {\n \"payload\": -433,\n \"topic\": \"M3 (Measured)\",\n \"mac\": \"3C1ACCBAA19D\",\n \"data_type\": \"int16\",\n \"property\": \"battery_power\",\n \"communication_status\": \"operational\",\n \"scale\": 1,\n \"unit\": \"W\",\n \"precision\": 1,\n \"friendly_name\": \"Battery Power\",\n \"device_class\": \"power\",\n \"state_class\": \"measurement\",\n \"entry_type\": \"SENSOR_DEFINITIONS\",\n \"_msgid\": \"16df34e86e841df2\",\n \"messageId\": \"6a3d7abd40cb157a90acc984\",\n \"modbusRequest\": {\n \"unitid\": 1,\n \"fc\": 3,\n \"address\": 30001,\n \"quantity\": 1,\n \"emptyMsgOnFail\": true,\n \"keepMsgProperties\": true,\n \"enableDeformedMessages\": false,\n \"messageId\": \"6a3d7abd40cb157a90acc984\"\n },\n \"responseBuffer\": {\n \"data\": [\n 65103\n ],\n \"buffer\": [\n 254,\n 79\n ]\n },\n \"parse_buffer\": [\n 254,\n 79\n ],\n \"specification\": {\n \"options\": {\n \"byteSwap\": false,\n \"resultType\": \"keyvalue\",\n \"singleResult\": true,\n \"msgProperty\": \"translated_payload\",\n \"setTopic\": true\n },\n \"items\": [\n {\n \"name\": \"value\",\n \"type\": \"int16be\",\n \"offset\": 0,\n \"length\": 1,\n \"mask\": \"\",\n \"scale\": \"1\",\n \"offsetbit\": 0,\n \"id\": 0,\n \"value\": -433\n }\n ]\n },\n \"originalPayload\": [\n 65103\n ],\n \"values\": [\n -433\n ],\n \"objectResults\": {\n \"value\": {\n \"name\": \"value\",\n \"type\": \"int16be\",\n \"offset\": 0,\n \"length\": 1,\n \"mask\": \"\",\n \"scale\": \"1\",\n \"offsetbit\": 0,\n \"id\": 0,\n \"value\": -433\n }\n },\n \"keyvalues\": {\n \"value\": -433\n },\n \"arrayResults\": [\n {\n \"name\": \"value\",\n \"type\": \"int16be\",\n \"offset\": 0,\n \"length\": 1,\n \"mask\": \"\",\n \"scale\": \"1\",\n \"offsetbit\": 0,\n \"id\": 0,\n \"value\": -433\n }\n ],\n \"buffer\": [\n 254,\n 79\n ],\n \"translated_payload\": {\n \"value\": -433\n },\n \"_event\": \"node:53ebc095797a50fc\"\n}\nmsg.payload = Math.floor(Math.random() * (2000 - (-1000) + 1)) + (-1000); \nfor (let i = 0; i < 10; i++) {\n msg.payload = Math.floor(Math.random() * (2000 - (-1000) + 1)) + (-1000); \n node.send(msg)\n} \nreturn null;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":350,"y":220,"wires":[["e8772e058c096989"]]},{"id":"1a80648df0d6abfd","type":"ui-chart","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","group":"4f87bd59a15b847e","name":"","label":"cpu test","order":9007199254740991,"chartType":"line","category":"topic","categoryType":"msg","xAxisLabel":"","xAxisProperty":"","xAxisPropertyType":"timestamp","xAxisType":"time","xAxisFormat":"","xAxisFormatType":"auto","xmin":"","xmax":"","yAxisLabel":"","yAxisProperty":"payload","yAxisPropertyType":"msg","ymin":"","ymax":"","bins":10,"action":"append","stackSeries":false,"pointShape":"circle","pointRadius":"1","showLegend":true,"removeOlder":"10","removeOlderUnit":"60","removeOlderPoints":"","colors":["#0095ff","#ff0000","#ff7f0e","#2ca02c","#a347e1","#d62728","#ff9896","#9467bd","#c5b0d5"],"textColor":["#666666"],"textColorDefault":true,"gridColor":["#e5e5e5"],"gridColorDefault":true,"width":6,"height":"4","className":"","interpolation":"step","x":860,"y":220,"wires":[[]]},{"id":"69c0edb80b6c835a","type":"inject","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"Block","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":350,"y":120,"wires":[["331341489f068df6"]]},{"id":"457d9fc3638fa678","type":"inject","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"Release","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":360,"y":160,"wires":[["0642db45d40d5b8f"]]},{"id":"331341489f068df6","type":"change","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"Set flow.block","rules":[{"t":"set","p":"block","pt":"flow","to":"true","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":520,"y":120,"wires":[[]]},{"id":"0642db45d40d5b8f","type":"change","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"Clear flow.block","rules":[{"t":"set","p":"block","pt":"flow","to":"false","tot":"bool"}],"action":"","property":"","from":"","to":"","reg":false,"x":520,"y":160,"wires":[[]]},{"id":"e8772e058c096989","type":"switch","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"Block?","property":"block","propertyType":"flow","rules":[{"t":"false"}],"checkall":"true","repair":false,"outputs":1,"x":510,"y":220,"wires":[["cc612ff604efa32f","1a80648df0d6abfd"]]},{"id":"cc612ff604efa32f","type":"debug","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"debug 16","active":false,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":640,"y":260,"wires":[]},{"id":"909073dac6635927","type":"function","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"reduce","func":"return {payload: msg.payload, topic: msg.topic}","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":670,"y":200,"wires":[["1a80648df0d6abfd"]]},{"id":"33366f32a7a526c7","type":"inject","z":"371f5f719cb5e2ab","g":"c12532a704294d5c","name":"Clear Chart","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[]","payloadType":"json","x":790,"y":120,"wires":[["1a80648df0d6abfd"]]},{"id":"4f87bd59a15b847e","type":"ui-group","name":"test","page":"c6ff182a4185f2f2","width":"6","height":"1","order":1,"showTitle":true,"className":"","visible":"true","disabled":"false","groupType":"default"},{"id":"c6ff182a4185f2f2","type":"ui-page","name":"Test page","ui":"a872b0a9bcc57d7d","path":"/testpage","icon":"home","layout":"grid","theme":"c4cba76c368d7996","breakpoints":[{"name":"Default","px":"0","cols":"3"},{"name":"Tablet","px":"576","cols":"6"},{"name":"Small Desktop","px":"768","cols":"9"},{"name":"Desktop","px":"1024","cols":"12"}],"order":10,"className":"","visible":true,"disabled":false},{"id":"a872b0a9bcc57d7d","type":"ui-base","name":"My Dashboard","path":"/dashboard","appIcon":"","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":false,"headerContent":"page","navigationStyle":"default","titleBarStyle":"default","showReconnectNotification":true,"notificationDisplayTime":1,"showDisconnectNotification":true,"allowInstall":false},{"id":"c4cba76c368d7996","type":"ui-theme","name":"Test theme","colors":{"surface":"#ffffff","primary":"#0094ce","bgPage":"#eeeeee","groupBg":"#ffffff","groupOutline":"#cccccc"},"sizes":{"density":"default","pagePadding":"12px","groupGap":"12px","groupBorderRadius":"4px","widgetGap":"12px"}},{"id":"2607f0bd5d70bbac","type":"global-config","env":[],"modules":{"@flowfuse/node-red-dashboard":"1.30.2"}}]
```

### Environment

- Dashboard version: 1.30.2 (also checked with 1.28.0)
- Node-RED version: 5.0.0
- Node.js version: 24.18.0
- npm version:
- Platform/OS: Ubunto 26.04 on laptop
- Browser:

### Have you provided an initial effort estimate for this issue?

I am not a FlowFuse team member

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the supplied reproduction flow and the ui-chart node, comparing messages that retain the additional properties with the reduced payload containing only payload and topic. Run the flow while blocking and releasing messages, then monitor Node-RED CPU usage over time. Done means identifying and fixing the CPU increase and confirming that chart updates remain stable with the full message.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.