influxdata / influxdata/kapacitor
UDF is shutdown immediately after measurements are imported
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
Hi,
I am testing the UDF [mirror.py](https://github.com/influxdata/kapacitor/blob/master/udf/agent/examples/mirror/mirror.py) function. When I import a file with about 2000 measurements in influxdb (influx -import -path=$file), the UDF is stopped after processing about 1000 measurements:
`[task_store] 2017/05/30 16:17:36 D! task normal_days finished
[task_master:main] 2017/05/30 16:17:36 E! Stopped task: normal_days mirror5: stopping UDF server: node aborted
[task_store] 2017/05/30 16:17:36 E! task normal_days finished with error: mirror5: stopping UDF server: node aborted
`
The error from mirror.py UDF:
`2017-05-30 16:17:35,550 INFO:root: Starting Agent for connection 3
Traceback (most recent call last):
File "/Users/.../code/udf/kapacitor/udf/agent.py", line 122, in _read_loop
self.handler.point(request.point)
File "/Users/.../code/udf/mirror.py", line 46, in point
self._agent.write_response(response, True)
File "/Users/.../code/udf/kapacitor/udf/agent.py", line 86, in write_response
self._out.flush()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
2017-05-30 16:17:36,216 ERROR:root: error processing request of type point: [Errno 32] Broken pipe
Exception in thread Thread-5:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "/Users/.../code/udf/mirror.py", line 61, in accept
a.wait()
File "/Users/.../code/udf/kapacitor/udf/agent.py", line 68, in wait
self._in.close()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 283, in close
self.flush()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 307, in flush
self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
`
For me it seems like Kapacitor is stopping the UDF immediately after all measurements are inserted into InfluxDB but not yet processed by the UDF (but not fully sure about this, just an assumption).
The "node aborted" error message is located [here](https://github.com/influxdata/kapacitor/blob/master/udf.go#L55)
macOS 10.12.5
both installed with homebrew:
Kapacitor v1.3.0 (git: unknown 7e27fd070e6a629a9b760c6300fade41364f4c7f)
InfluxDB vv1.2.4 (git: master 77909d7c7826afe597b12d957996d6e16cd1afaa)
Thanks in advance,
Simon
Contributor guide
Research direction
Start with udf.go around the "node aborted" error, then trace the UDF lifecycle through udf/agent.py, especially write_response and wait. Reproduce the import using udf/agent/examples/mirror/mirror.py and determine why processing stops after roughly half the measurements; done means the UDF remains available until the imported measurements are processed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100