influxdata / influxdata/kapacitor
Static scalar writeback and improvement of error handling and messaging
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
I tried to run a rather simple tickScript:
```javaScript
var aggregationPeriod = 60s
var myScalarOut = 10.0
var dbName = 'simple'
var retentionPolicy = 'autogen'
var outputMeasurementName = 'scalarValueTest'
var outputFieldName = 'value'
batch
|stats(aggregationPeriod)
.align()
|eval(lambda: myScalarOut ).as(outputFieldName).keep()
|influxDBOut()
.database(dbName)
.retentionPolicy(retentionPolicy)
.measurement(outputMeasurementName)
```
For computational purposes, we sometimes need timeseries with static values and an equidistant sampled time interval. From time to time, we change the "myScalarOut" as an output variable and let another value be written into the time series. Sometimes we need to update those times for a time in history. I know this should all be designed in a better way and not by reproducing the same value over and over again, but that is another thing.
The point here is:
This script doesn't work.
If I post it as a kapacitor task, I get in the kapacitor logs:
```
[task_master:main] 2017/07/25 11:44:43 I! Started task: dub-362530493
log messages must have 'L!' prefix where L is one of 'D', 'I', 'W', 'E'[log] 2017/07/25 11:44:43 http: panic serving 10.0.2.2:51490: interface conversion: kapacitor.Node is *kapacitor.NoOpNode, not *kapacitor.QueryNode
goroutine 6910 [running]:
net/http.(*conn).serve.func1(0xc420ad8f80)
/usr/local/go/src/net/http/server.go:1491 +0x12a
panic(0x1cec680, 0xc420ee0900)
/usr/local/go/src/runtime/panic.go:458 +0x243
github.com/influxdata/kapacitor.(*BatchNode).DBRPs(0xc420467f00, 0x1, 0x1, 0xc420ef5d70, 0xc4200227f8, 0xc420f30050)
/root/go/src/github.com/influxdata/kapacitor/batch.go:70 +0x244
github.com/influxdata/kapacitor.(*ExecutingTask).checkDBRPs(0xc420080870, 0xc420467f00, 0xc420f300b0, 0x1bb4d80)
/root/go/src/github.com/influxdata/kapacitor/task.go:291 +0x75
github.com/influxdata/kapacitor.(*ExecutingTask).StartBatching(0xc420080870, 0xc42004e940, 0xc420080870)
/root/go/src/github.com/influxdata/kapacitor/task.go:254 +0x68
github.com/influxdata/kapacitor/services/task_store.(*Service).startTask(0xc42053d720, 0xc4211863c0, 0xd, 0x1, 0xc4203ceba0, 0x1, 0x1, 0xc42052e540, 0x1b0, 0x0, ...)
/root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:1865 +0x1bf
github.com/influxdata/kapacitor/services/task_store.(*Service).handleCreateTask(0xc42053d720, 0x2e340e0, 0xc4203cea20, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:789 +0x8cb
github.com/influxdata/kapacitor/services/task_store.(*Service).(github.com/influxdata/kapacitor/services/task_store.handleCreateTask)-fm(0x2e340e0, 0xc4203cea20, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/task_store/service.go:130 +0x48
github.com/influxdata/kapacitor/services/httpd.authorize.func1(0x2e340e0, 0xc4203cea20, 0xc4200bd590, 0x1fba4d3, 0xa, 0x1, 0x2ea8360, 0x0, 0x0, 0xc4204c3f20)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:714 +0xe9
github.com/influxdata/kapacitor/services/httpd.authenticate.func1(0x2e340e0, 0xc4203cea20, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:579 +0xa6d
net/http.HandlerFunc.ServeHTTP(0xc420104f20, 0x2e340e0, 0xc4203cea20, 0xc4200bd590)
/usr/local/go/src/net/http/server.go:1726 +0x44
github.com/influxdata/kapacitor/services/httpd.jsonContent.func1(0x2e340e0, 0xc4203cea20, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:818 +0xb1
net/http.HandlerFunc.ServeHTTP(0xc420105120, 0x2e340e0, 0xc4203cea20, 0xc4200bd590)
/usr/local/go/src/net/http/server.go:1726 +0x44
github.com/influxdata/kapacitor/services/httpd.gzipFilter.func1(0x2e2bd60, 0xc4203ce9c0, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:811 +0x1e3
net/http.HandlerFunc.ServeHTTP(0xc4201051e0, 0x2e2bd60, 0xc4203ce9c0, 0xc4200bd590)
/usr/local/go/src/net/http/server.go:1726 +0x44
github.com/influxdata/kapacitor/services/httpd.versionHeader.func1(0x2e2bd60, 0xc4203ce9c0, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:827 +0xbc
net/http.HandlerFunc.ServeHTTP(0xc420105240, 0x2e2bd60, 0xc4203ce9c0, 0xc4200bd590)
/usr/local/go/src/net/http/server.go:1726 +0x44
github.com/influxdata/kapacitor/services/httpd.cors.func1(0x2e2bd60, 0xc4203ce9c0, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:860 +0xee
net/http.HandlerFunc.ServeHTTP(0xc420105260, 0x2e2bd60, 0xc4203ce9c0, 0xc4200bd590)
/usr/local/go/src/net/http/server.go:1726 +0x44
github.com/influxdata/kapacitor/services/httpd.requestID.func1(0x2e2bd60, 0xc4203ce9c0, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:870 +0x138
net/http.HandlerFunc.ServeHTTP(0xc420105280, 0x2e2bd60, 0xc4203ce9c0, 0xc4200bd590)
/usr/local/go/src/net/http/server.go:1726 +0x44
github.com/influxdata/kapacitor/services/httpd.logHandler.func1(0x2e2bd60, 0xc4203ce960, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:878 +0xfa
net/http.HandlerFunc.ServeHTTP(0xc4201052a0, 0x2e2bd60, 0xc4203ce960, 0xc4200bd590)
/usr/local/go/src/net/http/server.go:1726 +0x44
github.com/influxdata/kapacitor/services/httpd.recovery.func1(0x2e33e20, 0xc421204a90, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:887 +0xfa
net/http.HandlerFunc.ServeHTTP(0xc4201052e0, 0x2e33e20, 0xc421204a90, 0xc4200bd590)
/usr/local/go/src/net/http/server.go:1726 +0x44
github.com/influxdata/kapacitor/services/httpd.(*ServeMux).ServeHTTP(0xc4203e5980, 0x2e33e20, 0xc421204a90, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/mux.go:163 +0x7f
github.com/influxdata/kapacitor/services/httpd.(*Handler).ServeHTTP(0xc42043f400, 0x2e33e20, 0xc421204a90, 0xc4200bd590)
/root/go/src/github.com/influxdata/kapacitor/services/httpd/handler.go:365 +0xcf
net/http.serverHandler.ServeHTTP(0xc420349100, 0x2e33e20, 0xc421204a90, 0xc4200bd590)
/usr/local/go/src/net/http/server.go:2202 +0x7d
net/http.(*conn).serve(0xc420ad8f80, 0x2e360a0, 0xc4203c9640)
/usr/local/go/src/net/http/server.go:1579 +0x4b7
created by net/http.(*Server).Serve
/usr/local/go/src/net/http/server.go:2293 +0x44d
```
and even worse: My REST Response in my Java client is:
```
java.net.ConnectException: Connection refused: connect
at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method)
at java.net.DualStackPlainSocketImpl.socketConnect(DualStackPlainSocketImpl.java:85)
```
Sometimes, I get an error response which states that the task would already exist even though, I created it with a random number and just restarted my kapacitor docker image (thus having no task in it at all!).
Even more intersting: Even though there is an error, the script runs just fine. On the command line, I can do a `kapacitor list tasks` and see that task as enabled. If I query my influx DB, it is filled with my data points continously as I wish.
But most interesting is:
If I delete this task, I am kind of going to destro kapacitor.
I get a `java.net.SocketTimeoutException: Read timed out` and afterwards, I cannot list tasks anymore. If I do `kapacitor list tasks` on the commandline, it just gets stuck and won't result anything anymore. All REST requests (at least creating tasks, listing tasks, deleting tasks, but not ping!) will fail from that point on!
I reproduced this issue several times: I resetup all docker images ( I use `https://github.com/influxdata/TICK-docker.git` from the `1.2` folder ) and I restarted my machine several times, as I first thought, there might be a networking issue before I looked into the kapacitor logs and saw that error.
My question is:
Is there any way I can write a static value (from a variable) continuously into InfluxDB from within Kapacitor? I don't want to mix up my scripts such that some of them are in InfluxDB (Continuous Queries), some of them are in Kapacitor, and some stuff, I do on my own. I would really like to have Kapacitor doing all my computations. And of course, I don't want to perform a dummy query, just to do something even though I don't need anything...
And my request:
I would be rather happy to first get an useful error message by creating an errorneous task like this and second, don't let an invalid script be able to make kapacitor not working correctly anymore.
Contributor guide
Research direction
Start with the panic at batch.go:70 and follow the task creation and deletion paths through task.go and services/task_store/service.go. Reproduce the supplied batch task, then verify that invalid task input returns a useful error without panicking, hanging REST requests, or leaving task management unusable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100