influxdata / influxdata/influxdb
Tick scripts generates values from older dates for percentile calculation
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
We are using Kapacitor scripts to perform transformation on our measurement,
Sample script:
stream
|from()
.measurement('tracking_timeTaken')
|delete()
.field('timeTaken41')
.field('timeTaken42')
.field('id')
.field('document')
.tag('checkpoint')
|window()
.period(5m)
.every(5m)
|groupBy('itemType')
|percentile('processingTime', 95.0)
|log()
|influxDBOut()
.database(db_name)
.retentionPolicy(retention_policy)
.measurement('tracking_95p')
The measurement 'tracking_timeTaken' looks like
time itemType document id checkpoint timeTaken41 timeTaken42 processingTime
2023-11-14T14:51:30.442+05:30 docType1 docType1 testdata12345 inprocess 0 0 6285
2023-11-14T14:51:31.470+05:30 docType1 docType1 testdata12346 open 0 0 7309
2023-11-14T15:08:26.118+05:30 docType1 docType1 testdata12347 inprocess 0 0 4428
2023-11-14T15:08:27.128+05:30 docType1 docType1 testdata12348 completed 0 0 5404
2023-11-14T14:55:11.012+05:30 docType2 docType2 testdata12349 inprocess 0 0 3400
2023-11-15T01:54:13.956+05:30 docType2 docType2 testdata12350 inprocess 0 0 11443
2023-11-15T02:00:56.706+05:30 docType2 docType2 testdata12351 error 0 0 4779
Expected behaviour: For itemType docType1 the response measurement tracking_95p considering 5mins window should contain values
time itemType percentile
2023-11-14T15:08:27.128+05:30 docType1 5404
2023-11-14T14:51:31.470+05:30 docType1 7309
Actual behaviour: However, in response we are getting older timestamps which are not present in source measurement
time itemType percentile
2023-11-13T15:14:44.015+05:30 docType1 6285
2023-11-14T02:05:56.706+05:30 docType1 7309
2023-11-14T15:14:45.025+05:30 docType1 7309
Here the time, '2023-11-13T15:14:44.015+05:30' is no where present in the source table and also 7309 is repeated.
Please share feedback and resolution tips
Contributor guide
Research direction
Start with the supplied Kapacitor stream and compare the source timestamps with the window and percentile output. Reproduce the repeated values and unexpected timestamps using the example data, then trace how the window and percentile stages assign output times. Done means identifying the cause and documenting a verified correction or limitation.
Written by the indexing model from the issue text.
Assessment
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100