influxdata / influxdata/kapacitor

Kapacitor record query converts integer to float

Open
#1,338 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

Consider data in measurement `m` like so:

```
time data
1 5
2 6
3 4
```

If `data` has a integer field type, then the following TICK script creates corresponding entries with integer field type when enabled and running on live data:

```
stream
|from()
.measurement('m')
|shift(1ns)
|influxDBOut()
.measurement('future_m')
```

However, to run on historical data, one must make and replay a recording like so:

```
RID=$(kapacitor record query -query $'SELECT data FROM "d"."autogen"."m" WHERE time >= \'2017-04-23T00:00:00Z\' AND time < \'2017-04-24T00:00:00Z\' GROUP BY * ' -type stream)
kapacitor replay -task my_task -recording $RID -rec-time
```

Maddeningly, this does not work, because the replayed data converts the integer field into a float, which is not compatible with an integer field of the same name. In fact, surprisingly, it creates another field, with the same name, forever preventing the live script from working.

No manner of reordering replays/live data or even casting using `::integer` in the query, makes any difference. I'm tempted to change everything to float because I understand integer is a bit of a hangover, but I have quite a bit of architecture built around these integer measurements. I just wish kapacitor would return queries the same as influx. Every time I go to do this I'm caught out by differences.

In the end I found this nasty workaround:

```
stream
|from()
.measurement('m')
|shift(1ns)
|eval(lambda: int("data"))
.as('data')
|influxDBOut()
.measurement('future_m')
```

which seems to work for now.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with the shown `kapacitor record query` and `kapacitor replay` commands, comparing the field type with live data. Trace the record-query and replay paths, then verify that integer fields remain integer-compatible and that the TICK script writes without requiring the `eval(lambda: int("data"))` workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.