influxdata / influxdata/kapacitor
UDF function cannot be used safely after join node with fill(null)
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
When you use join node with fill(null) it is possible that one of the joined streams will be missed and then Kapacitor will suggest that this stream has the same fields as in another stream and add them to it with values==null, if you use UDF node after this it will cause panic error and Kapacitor restart because Kapacitor cannot serialise null values to send them to UDF function (kapacitor/blob/master/udf/server.go, function fieldsToTypedMaps).
It seems like there cases when null values are meaningful for a UDF function and should be passed there, so I think their serialisation should be supported.
P.S.
Suggestion that missed stream has the same fields as another stream seems to be wrong at all, I think this logic should be changed as well. Why it is impossible to not fill missed stream with any fields (required fields may be added later with default node for example)?
Contributor guide
Research direction
Start in udf/server.go at fieldsToTypedMaps and trace how join with fill(null) produces fields before values are sent to a UDF. Reproduce the missed-stream case, then verify that null values can be serialized without a panic or restart; the issue also raises a separate question about changing the fill behavior, which needs a maintainer decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100