influxdata / influxdata/kapacitor

Kapacitor UDF join problem

Open
#2,330 0 comments 0 reactions 0 assignees View on GitHub
pm/review
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

Hi all,
I need some help with Kapacitor UDF (python). I’ve got implementation, seems like it works, i can log data from, and i see that data is correct. Here is tick blob of it:

var online_users_rate = online_users_count
|join(online_users_diff)
.as(‘count’, ‘diff’)
.tolerance(1s)
|eval(lambda: “diff.value” / “count.value”)
.as(‘value’)

var online_users_rate_hpf = online_users_rate
@hpf()
.field(‘value’)
.warmup(0)
.cutoff(0.0001)
.as(‘value’)

online_users_diff and online_users_count - are batches.

When i’m trying to join:

online_users_rate
|join(online_users_rate_hpf)
.as(‘rate’, ‘rate_hpf’)
.tolerance(1s)
|log()

it failed, there is no results, but if i try to join like

online_users_rate_hpf
|join(online_users_rate_hpf)
.as(‘rate’, ‘rate_hpf’)
.tolerance(1s)
|log()

it works. So joining results of my @hpf UDF with it self works, joining with something - doesn’t.

One side notice, i did the following logs from the python agent:

def begin_batch(self, begin_req):
logger.info("Starting batch {}".format(begin_req))
self._input_points = []
self._filter_state.reset()

# Keep copy of begin_batch
response = udf_pb2.Response()
response.begin.CopyFrom(begin_req)
self._begin_response = response

and in log i see the following:

ts=2020-04-29T04:03:54.278Z lvl=info msg=“UDF log” service=kapacitor task_master=main task=test node=hpf6 text=“INFO:root:Starting batch name: “online_users””

Where it’s getting batch name from ??

Please i urgently need help, trying to push it to the prod like new tech stack, but without custom UDF - there is no chance, from example above hpf - it’s high pass filter.

Contributor guide

Open the contributing guide

Research direction

Start with the supplied TICK examples and the Python UDF begin_batch implementation, then trace how batch names and timestamps are handled before the join. Reproduce the differing join results between online_users_rate and online_users_rate_hpf; done means identifying the cause of the missing results and establishing the expected UDF join behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.