influxdata / influxdata/kapacitor

UDF function cannot be used safely after join node with fill(null)

Open
#2,466 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

When you use join node with fill(null) it is possible that one of the joined streams will be missed and then Kapacitor will suggest that this stream has the same fields as in another stream and add them to it with values==null, if you use UDF node after this it will cause panic error and Kapacitor restart because Kapacitor cannot serialise null values to send them to UDF function (kapacitor/blob/master/udf/server.go, function fieldsToTypedMaps).
It seems like there cases when null values are meaningful for a UDF function and should be passed there, so I think their serialisation should be supported.

P.S.
Suggestion that missed stream has the same fields as another stream seems to be wrong at all, I think this logic should be changed as well. Why it is impossible to not fill missed stream with any fields (required fields may be added later with default node for example)?

Contributor guide

Open the contributing guide

Research direction

Start in udf/server.go at fieldsToTypedMaps and trace how join with fill(null) produces fields before values are sent to a UDF. Reproduce the missed-stream case, then verify that null values can be serialized without a panic or restart; the issue also raises a separate question about changing the fill behavior, which needs a maintainer decision.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.