Filebeat netflow GC "collecting" 402GB of RAM

Open

#14,917 opened on Dec 3, 2019 13 comments 0 reactions 1 assignee Claimed by @adriansr View on GitHub

Filebeat Team:Integrations

Dominant language: Go
Stars: 12.7k
Forks: 5k
Avg merge: 2d 15m
Merged PRs (30d): 385

Description

For confirmed bugs, please report:
- Version: 7.5.0
- Operating System: CentOS 7, filebeat installed with yum

So I am not quite sure if this will be easy to reproduce. I am writing this issue as I am debugging a weird RAM behavior while tinkering with Netflow.
We have around 1000-1500 flows per second, maxing out at 3000, deployed filebeat 7.5.0 with NetFlow module, on a CentOS 7, 8 cores, 8 GB RAM. That filebeat is receiving some ASA events, so there might be a ground noise around 200-500 events per second.
My three-node ES cluster with 7.5.0 and Kibana 7.5.0. My ES cluster can handle around 14-16k documents per second on ingest. So backpressure should not be any problem.
Somewhere around 14:30 today, CET, we started with the filebeat and NetFlow module, with the following configuration.

Super long config!

```
filebeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml

# Set to true to enable config reloading
reload.enabled: false

# Period on which files under path should be checked for changes
#reload.period: 10s

setup.template.settings:
index.number_of_shards: 1

filebeat.inputs:
- type: syslog
protocol.udp:
host: "0.0.0.0:514"
filebeat.modules:
- module: netflow
log:
enabled: true
var:
netflow_host: 0.0.0.0
netflow_port: 2055
max_message_size: 1000
queue_size: 32768
#-------- Elasticsearch output ------------------------------
output.elasticsearch:
hosts: ["es1:9200","es2:9200","es3:9200"]
protocol: "https"
username: ""
password: ""
ssl.certificate_authorities: ["/etc/ssl/certs/cachain.pem"]
pipeline: geoip-info

processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
- dns:
type: reverse
action: append
fields:
server.ip: server.hostname
client.ip: client.hostname
source.ip: source.hostname
destination.ip: destination.hostname
success_cache:
capacity.initial: 10000
capacity.max: 100000
failure_cache:
capacity.initial: 1000
capacity.max: 10000
ttl: 1m
nameservers: ['1.1.1.1', 'someinternal']
timeout: 5s
tag_on_failure: [_dns_reverse_lookup_failed]
- add_fields:
when.network.source.ip: 192.132.0.0/16
fields:
source.geo.location:
lat: 1.377502
lon: 103.872831
target: ''
- add_fields:
when.network.source.ip: 192.61.0.0/16
fields:
source.geo.location:
lat: 51.200045
lon: 4.435278
target: ''
- add_fields:
when.network.source.ip: 192.151.0.0/16
fields:
source.geo.location:
lat: 39.923802
lon: 116.415285
target: ''
- add_fields:
when.network.source.ip: 192.137.0.0/16
fields:
source.geo.location:
lat: 14.169850
lon: 121.132613
target: ''
- add_fields:
when.network.source.ip: 192.64.0.0/16
fields:
source.geo.location:
lat: 52.228267
lon: 0.157753
target: ''
- add_fields:
when.network.source.ip: 192.23.0.0/16
fields:
source.geo.location:
lat: 37.321643
lon: -122.032561
target: ''
- add_fields:
when.network.source.ip: 192.45.0.0/16
fields:
source.geo.location:
lat: 51.411013
lon: 5.450380
target: ''
- add_fields:
when.network.source.ip: 192.63.0.0/16
fields:
source.geo.location:
lat: 32.659898
lon: -16.926293
target: ''
- add_fields:
when.network.source.ip: 192.139.0.0/16
fields:
source.geo.location:
lat: 17.434485
lon: 78.387469
target: ''
- add_fields:
when.network.source.ip: 192.65.0.0/16
fields:
source.geo.location:
lat: 50.891908
lon: 11.594972
target: ''
- add_fields:
when.network.source.ip: 192.129.0.0/16
fields:
source.geo.location:
lat: 1.339557
lon: 103.708979
target: ''
- add_fields:
when.network.source.ip: 192.36.10.0/24
fields:
source.geo.location:
lat: 46.101176
lon: 7.074462
target: ''
- add_fields:
when.network.source.ip: 192.26.0.0/16
fields:
source.geo.location:
lat: 40.232804
lon: -74.687870
target: ''
- add_fields:
when.network.source.ip: 192.32.0.0/16
fields:
source.geo.location:
lat: 45.463651
lon: 9.197683
target: ''
- add_fields:
when.network.source.ip: 192.40.10.0/24
fields:
source.geo.location:
lat: 48.111018
lon: 11.450526
target: ''
- add_fields:
when.network.source.ip: 192.62.0.0/16
fields:
source.geo.location:
lat: 47.721335
lon: 6.085081
target: ''
- add_fields:
when.network.source.ip: 192.33.0.0/16
fields:
source.geo.location:
lat: 45.194482
lon: 9.141818
target: ''
- add_fields:
when.network.source.ip: 192.31.0.0/16
fields:
source.geo.location:
lat: 43.684821
lon: 10.487538
target: ''
- add_fields:
when.network.source.ip: 192.20.0.0/16
fields:
source.geo.location:
lat: 43.684837
lon: 10.487381
target: ''
- add_fields:
when.network.source.ip: 192.35.10.0/24
fields:
source.geo.location:
lat: 50.389876
lon: -4.020370
target: ''
- add_fields:
when.network.source.ip: 192.11.0.0/16
fields:
source.geo.location:
lat: 46.975885
lon: 15.394378
target: ''
- add_fields:
when.network.source.ip: 192.251.0.0/16
fields:
source.geo.location:
lat: 46.975885
lon: 15.394378
target: ''
- add_fields:
when.network.source.ip: 192.17.0.0/16
fields:
source.geo.location:
lat: 46.975885
lon: 15.394378
target: ''
- add_fields:
when.network.source.ip: 192.10.0.0/16
fields:
source.geo.location:
lat: 46.975885
lon: 15.394378
target: ''
- add_fields:
when.network.source.ip: 192.12.0.0/16
fields:
source.geo.location:
lat: 46.975885
lon: 15.394378
target: ''
- add_fields:
when.network.source.ip: 192.14.0.0/16
fields:
source.geo.location:
lat: 46.975885
lon: 15.394378
target: ''
- add_fields:
when.network.source.ip: 192.16.0.0/16
fields:
source.geo.location:
lat: 46.975885
lon: 15.394378
target: ''
- add_fields:
when.network.source.ip: 192.168.0.0/16
fields:
source.geo.location:
lat: 46.975885
lon: 15.394378
target: ''
- add_fields:
when.network.source.ip: 192.19.0.0/16
fields:
source.geo.location:
lat: 46.975885
lon: 15.394378
target: ''
- add_fields:
when.network.source.ip: 192.34.0.0/16
fields:
source.geo.location:
lat: 47.224153
lon: 8.817450
target: ''
- add_fields:
when.network.source.ip: 192.43.0.0/16
fields:
source.geo.location:
lat: 48.496135
lon: 9.137428
target: ''
- add_fields:
when.network.source.ip: 192.38.0.0/16
fields:
source.geo.location:
lat: 47.303424
lon: 8.546351
target: ''
- add_fields:
when.network.source.ip: 192.27.0.0/16
fields:
source.geo.location:
lat: 37.385708
lon: -121.972477
target: ''
- add_fields:
when.network.source.ip: 192.148.0.0/16
fields:
source.geo.location:
lat: 37.556852
lon: 126.981990
target: ''
- add_fields:
when.network.source.ip: 192.152.0.0/16
fields:
source.geo.location:
lat: 31.228960
lon: 121.524242
target: ''
- add_fields:
when.network.source.ip: 192.154.0.0/16
fields:
source.geo.location:
lat: 22.534638
lon: 114.054346
target: ''
- add_fields:
when.network.source.ip: 192.66.0.0/16
fields:
source.geo.location:
lat: 51.659808
lon: -0.914749
target: ''
- add_fields:
when.network.source.ip: 192.44.0.0/16
fields:
source.geo.location:
lat: 49.079522
lon: 8.458652
target: ''
- add_fields:
when.network.source.ip: 192.153.0.0/16
fields:
source.geo.location:
lat: 31.298766
lon: 120.585207
target: ''
- add_fields:
when.network.source.ip: 192.144.0.0/16
fields:
source.geo.location:
lat: 25.081430
lon: 121.569353
target: ''
- add_fields:
when.network.source.ip: 192.134.0.0/16
fields:
source.geo.location:
lat: 1.377595
lon: 103.872874
target: ''
- add_fields:
when.network.source.ip: 192.135.0.0/16
fields:
source.geo.location:
lat: 1.377595
lon: 103.872874
target: ''
- add_fields:
when.network.source.ip: 192.146.0.0/16
fields:
source.geo.location:
lat: 1.111222
lon: 123.123123
target: ''
- add_fields:
when.network.source.ip: 192.37.0.0/16
fields:
source.geo.location:
lat: 39.479420
lon: -0.334650
target: ''
- add_fields:
when.network.source.ip: 192.41.0.0/16
fields:
source.geo.location:
lat: 52.326310
lon: 13.626244
target: ''
setup.kibana.host: "https://kibana"
setup.kibana.ssl.enabled: true
setup.kibana.ssl.certificate_authorities: ["/etc/ssl/certs/cachain.pem"]

monitoring:
enabled: true
elasticsearch:
username: beats_system
password:
```

I sometimes had this message.

```
Dec 03 18:46:52 hostname filebeat[32543]: 2019-12-03T18:46:52.743+0100 WARN [netflow] netflow/input.go:240 Error parsing NetFlow packet of length 1000 from 10.12.0.110:56398: error reading header: EOF
Dec 03 18:46:53 sux7488.amsint.com filebeat[32543]: 2019-12-03T18:46:53.742+0100 WARN [netflow] netflow/input.go:240 Error parsing NetFlow packet of length 1000 from 10.12.0.110:56398: error reading header: EOF
Dec 03 18:47:07 hostname.
```

So I altered the `max_message_size: 1000` to `max_message_size: 10000` which worked. The funny thing is, if I watch the beat in monitoring, I am confused with the values I see.

Since 14:30, it used some memory. Ok in that graph you can not see it very well due to the peak at ~ 18:00.
![10 11 107 89_5601_app_monitoring](https://user-images.githubusercontent.com/12175559/70078750-8f1d5a80-1603-11ea-9aee-97d7edf003f0.png)

If I scale down the time to 14:00 - 14:30, you can see that at 14:18, the first flows started coming in.
![10 11 107 89_5601_app_monitoring (2)](https://user-images.githubusercontent.com/12175559/70078785-a4928480-1603-11ea-8877-c8f18283db4a.png)

If I move mor to 14:10 - 15, filebeat took so much memory around 8gb?
![10 11 107 89_5601_app_monitoring (1)](https://user-images.githubusercontent.com/12175559/70078825-bffd8f80-1603-11ea-9670-c3ddbda9234f.png)

Around 18:00, I changed the configuration to include all the `source.geo` lookups also to include `destination.geo`, so I just copied my `add_fields` from above and added those. As soon as I have done this, and restarted the filebeat, it acted out crazy.

![10 11 107 89_5601_app_monitoring (3)](https://user-images.githubusercontent.com/12175559/70078931-f6d3a580-1603-11ea-83a9-a33ff9437358.png)

![10 11 107 89_5601_app_monitoring (5)](https://user-images.githubusercontent.com/12175559/70078947-fc30f000-1603-11ea-8966-69d0a2066c9b.png)

Also at the time, I saw some super weird stuff in the `journalctl` logs.

Super long and weird journalctl

```
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: fatal error: runtime: out of memory
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime stack:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.throw(0x300364b, 0x16)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/panic.go:617 +0x72
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.sysMap(0xfea4000000, 0x4000000, 0x4e7c5f8)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/mem_linux.go:170 +0xc7
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.(*mheap).sysAlloc(0x4e62a20, 0x9c4000, 0x4e62a30, 0x4e2)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/malloc.go:633 +0x1cd
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.(*mheap).grow(0x4e62a20, 0x4e2, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/mheap.go:1222 +0x42
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.(*mheap).allocSpanLocked(0x4e62a20, 0x4e2, 0x4e7c608, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/mheap.go:1150 +0x37f
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.(*mheap).alloc_m(0x4e62a20, 0x4e2, 0x101, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/mheap.go:977 +0xc2
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.(*mheap).alloc.func1()
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/mheap.go:1048 +0x4c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.(*mheap).alloc(0x4e62a20, 0x4e2, 0x7f56ad010101, 0x12a60e0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/mheap.go:1047 +0x8a
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.largeAlloc(0x9c4000, 0x101, 0x4c37980)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/malloc.go:1055 +0x99
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.mallocgc.func1()
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/malloc.go:950 +0x46
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.systemstack(0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/asm_amd64.s:351 +0x66
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.mstart()
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/proc.go:1153
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 315 [running]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.systemstack_switch()
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/asm_amd64.s:311 fp=0xc00094ee28 sp=0xc00094ee20 pc=0x12db2a0
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.mallocgc(0x9c4000, 0x2bef200, 0xc00094ee01, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/malloc.go:949 +0x872 fp=0xc00094eec8 sp=0xc00094ee28 pc=0x128b8f2
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.makeslice(0x2bef200, 0x9c4000, 0x9c4000, 0x33f7200)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/slice.go:49 +0x6c fp=0xc00094eef8 sp=0xc00094eec8 pc=0x12c452c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).run(0xc00005b5c0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:98 +0x81 fp=0xc00094efb0 sp=0xc00094eef8 pc=0x25c2f51
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start.func1(0xc00005b5c0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:85 +0x57 fp=0xc00094efd8 sp=0xc00094efb0 pc=0x25c3477
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: runtime.goexit()
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc00094efe0 sp=0xc00094efd8 pc=0x12dd371
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:83 +0x172
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 1 [chan receive, 2 minutes]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/beater.(*signalWait).Wait(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/beater/signalwait.go:45
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/beater.(*Filebeat).Run(0xc0008b15a0, 0xc000259d40, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/beater/filebeat.go:410 +0xf13
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/cmd/instance.(*Beat).launch(0xc000259d40, 0x2fe465b, 0x8, 0x2fe465b, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:435 +0x60e
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/cmd/instance.Run.func1(0x2fe465b, 0x8, 0x2fe465b, 0x8, 0x0, 0x0, 0xc0007ffc90, 0x30d5cf0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:187 +0x536
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/cmd/instance.Run(0x2fe465b, 0x8, 0x2fe465b, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/cmd/instance/beat.go:188 +0x13a
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/cmd.genRunCmd.func1(0xc000286000, 0xc0001308f0, 0x0, 0xb)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/cmd/run.go:36 +0x80
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/github.com/spf13/cobra.(*Command).execute(0xc000286000, 0xc00004e0d0, 0xb, 0xb, 0xc000286000, 0xc00004e0d0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:704 +0x289
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0xc000286000, 0x2a0d419, 0xc0005e7f88, 0xc0000be058)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:785 +0x2ca
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/github.com/spf13/cobra.(*Command).Execute(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/github.com/spf13/cobra/command.go:738
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: main.main()
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/main.go:22 +0x30
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 20 [syscall, 2 minutes]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: os/signal.signal_recv(0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/sigqueue.go:139 +0x9c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: os/signal.loop()
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/os/signal/signal_unix.go:23 +0x22
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by os/signal.init.0
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/os/signal/signal_unix.go:29 +0x41
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 21 [chan receive]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/k8s.io/klog.(*loggingT).flushDaemon(0x4e5ad40)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/k8s.io/klog/klog.go:1000 +0x8b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/vendor/k8s.io/klog.init.0
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/k8s.io/klog/klog.go:404 +0x6c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 11 [select]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/go.opencensus.io/stats/view.(*worker).start(0xc000050190)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/go.opencensus.io/stats/view/worker.go:154 +0x100
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/vendor/go.opencensus.io/stats/view.init.0
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/go.opencensus.io/stats/view/worker.go:32 +0x57
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 1127 [select]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*persistConn).writeLoop(0xc003d9c6c0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1978 +0x113
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1360 +0xb0d
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 497 [IO wait]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d96dd8, 0x72, 0xffffffffffffffff)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc00bcb1618, 0x72, 0xe700, 0xe78e, 0xffffffffffffffff)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: internal/poll.(*FD).Read(0xc00bcb1600, 0xc1dbc56000, 0xe78e, 0xe78e, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net.(*netFD).Read(0xc00bcb1600, 0xc1dbc56000, 0xe78e, 0xe78e, 0x203000, 0x0, 0xe781)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:202 +0x4f
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net.(*conn).Read(0xc00018a110, 0xc1dbc56000, 0xe78e, 0xe78e, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/net.go:177 +0x69
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*atLeastReader).Read(0xc000780900, 0xc1dbc56000, 0xe78e, 0xe78e, 0x40, 0x2534, 0xc003d618d8)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:761 +0x60
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: bytes.(*Buffer).ReadFrom(0xc000325758, 0x33cb6c0, 0xc000780900, 0x1289475, 0x2cf3640, 0x2f21b60)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/bytes/buffer.go:207 +0xbd
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readFromUntil(0xc000325500, 0x33cfae0, 0xc00018a110, 0x5, 0xc00018a110, 0x40)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:783 +0xf8
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecordOrCCS(0xc000325500, 0x30da000, 0xc000325638, 0xc003d61b38)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:590 +0x125
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecord(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:558
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).Read(0xc000325500, 0xc00fd5a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:1236 +0x137
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/transport.(*statsConn).Read(0xc00fcc3140, 0xc00fd5a000, 0x1000, 0x1000, 0xc003d61c50, 0x1284b8c, 0xc00fc01980)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/transport/stats.go:44 +0x55
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*persistConn).Read(0xc00bdab8c0, 0xc00fd5a000, 0x1000, 0x1000, 0xc003d61c88, 0x1284665, 0xc00fc01980)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1526 +0x7b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: bufio.(*Reader).fill(0xc0007a6a20)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:100 +0x10f
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: bufio.(*Reader).Peek(0xc0007a6a20, 0x1, 0x0, 0x0, 0x1, 0xc00bd90d00, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:138 +0x4f
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*persistConn).readLoop(0xc00bdab8c0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1679 +0x1a3
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1359 +0xae8
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 1126 [IO wait]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d96f78, 0x72, 0xffffffffffffffff)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc00022e498, 0x72, 0xe700, 0xe78e, 0xffffffffffffffff)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: internal/poll.(*FD).Read(0xc00022e480, 0xc00fe86000, 0xe78e, 0xe78e, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net.(*netFD).Read(0xc00022e480, 0xc00fe86000, 0xe78e, 0xe78e, 0x203000, 0x0, 0xe781)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:202 +0x4f
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net.(*conn).Read(0xc000784ac0, 0xc00fe86000, 0xe78e, 0xe78e, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/net.go:177 +0x69
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*atLeastReader).Read(0xc000781300, 0xc00fe86000, 0xe78e, 0xe78e, 0x40, 0x2d53, 0xc003d658d8)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:761 +0x60
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: bytes.(*Buffer).ReadFrom(0xc0006493d8, 0x33cb6c0, 0xc000781300, 0x1289475, 0x2cf3640, 0x2f21b60)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/bytes/buffer.go:207 +0xbd
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readFromUntil(0xc000649180, 0x33cfae0, 0xc000784ac0, 0x5, 0xc000784ac0, 0x40)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:783 +0xf8
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecordOrCCS(0xc000649180, 0x30da000, 0xc0006492b8, 0xc003d65b38)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:590 +0x125
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecord(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:558
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).Read(0xc000649180, 0xc0000fc000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:1236 +0x137
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/transport.(*statsConn).Read(0xc00bdc6fe0, 0xc0000fc000, 0x1000, 0x1000, 0xc003d65c50, 0x1284b8c, 0xc007ee69c0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/transport/stats.go:44 +0x55
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*persistConn).Read(0xc003d9c6c0, 0xc0000fc000, 0x1000, 0x1000, 0xc003d65c88, 0x1284665, 0xc007ee69c0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1526 +0x7b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: bufio.(*Reader).fill(0xc001eb80c0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:100 +0x10f
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: bufio.(*Reader).Peek(0xc001eb80c0, 0x1, 0x0, 0x0, 0x1, 0xc00bd90f00, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:138 +0x4f
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*persistConn).readLoop(0xc003d9c6c0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1679 +0x1a3
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1359 +0xae8
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 278 [select]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/queue/memqueue.(*bufferingEventLoop).run(0xc000170d20)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/eventloop.go:316 +0x1db
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/queue/memqueue.NewBroker.func1(0xc00047a4d0, 0x33f0ac0, 0xc000170d20)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:175 +0x5d
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/queue/memqueue.NewBroker
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:173 +0x374
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 279 [select]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/queue/memqueue.(*ackLoop).run(0xc000150d70)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/ackloop.go:60 +0x107
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/queue/memqueue.NewBroker.func2(0xc00047a4d0, 0xc000150d70)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:179 +0x57
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/queue/memqueue.NewBroker
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:177 +0x3a3
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 280 [select]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*eventConsumer).loop(0xc000150e60, 0x33f0b00, 0xc0008b0000)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/consumer.go:190 +0x2c9
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.newEventConsumer
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/consumer.go:77 +0x155
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 281 [select, 2 minutes]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*retryer).loop(0xc000150eb0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/retry.go:139 +0x1df
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.newRetryer
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/retry.go:91 +0x13d
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 282 [select]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*persistConn).roundTrip(0xc003d9c6c0, 0xc01bdb2cc0, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:2194 +0x75c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Transport).roundTrip(0xc000857200, 0xc0007bc800, 0xc01bdb2c90, 0xd597d66d8c, 0xd597d66d80)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:481 +0xa1b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Transport).RoundTrip(0xc000857200, 0xc0007bc800, 0xc000857200, 0xbf71c3628ed793f3, 0x2be91491df)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/roundtrip.go:17 +0x35
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.send(0xc0007bcb00, 0x33cfba0, 0xc000857200, 0xbf71c3628ed793f3, 0x2be91491df, 0x4e5a700, 0xc000010630, 0xbf71c3628ed793f3, 0x1, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:250 +0x461
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Client).send(0xc0005bb9b0, 0xc0007bcb00, 0xbf71c3628ed793f3, 0x2be91491df, 0x4e5a700, 0xc000010630, 0x0, 0x1, 0x10)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:174 +0xfb
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Client).do(0xc0005bb9b0, 0xc0007bcb00, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:641 +0x279
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Client).Do(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:509
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Connection).execHTTPRequest(0xc00023ed80, 0xc0007bcb00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:826 +0x34c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Connection).sendBulkRequest(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/bulkapi.go:203
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Client).publishEvents(0xc00023ed80, 0xc01baf0000, 0x32, 0x800, 0xc1dbaad0e0, 0xc1dbaad0e0, 0x0, 0xc00023f680, 0xc00023f708)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:342 +0x341
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Client).Publish(0xc00023ed80, 0x3448080, 0xc0005b6780, 0xc000802d20, 0xc000799f78)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:293 +0x43
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs.(*backoffClient).Publish(0xc0005bb9e0, 0x3448080, 0xc0005b6780, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/backoff.go:60 +0x4b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*netClientWorker).run(0xc0007ed5c0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:119 +0x315
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.makeClientWorker
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:48 +0xe0
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 283 [select]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*persistConn).roundTrip(0xc00fcc57a0, 0xd467c45620, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:2194 +0x75c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Transport).roundTrip(0xc000857440, 0xc00077e700, 0xd467c455f0, 0xc01bd2c59c, 0xc01bd2c590)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:481 +0xa1b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Transport).RoundTrip(0xc000857440, 0xc00077e700, 0xc000857440, 0xbf71c362499583b9, 0x2ba837b7a7)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/roundtrip.go:17 +0x35
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.send(0xc0007bce00, 0x33cfba0, 0xc000857440, 0xbf71c362499583b9, 0x2ba837b7a7, 0x4e5a700, 0xc00081c1a8, 0xbf71c362499583b9, 0x1, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:250 +0x461
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Client).send(0xc0005bbc20, 0xc0007bce00, 0xbf71c362499583b9, 0x2ba837b7a7, 0x4e5a700, 0xc00081c1a8, 0x0, 0x1, 0x10)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:174 +0xfb
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Client).do(0xc0005bbc20, 0xc0007bce00, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:641 +0x279
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Client).Do(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:509
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Connection).execHTTPRequest(0xc00023ef00, 0xc0007bce00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:826 +0x34c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Connection).sendBulkRequest(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/bulkapi.go:203
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Client).publishEvents(0xc00023ef00, 0xc013fc8000, 0x2c, 0x800, 0xc0008f4300, 0xc0008f4300, 0x0, 0xc00023f800, 0xc00023f888)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:342 +0x341
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Client).Publish(0xc00023ef00, 0x3448080, 0xc0005b66c0, 0xc000802d20, 0xc00fe5df78)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:293 +0x43
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs.(*backoffClient).Publish(0xc0005bbc50, 0x3448080, 0xc0005b66c0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/backoff.go:60 +0x4b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*netClientWorker).run(0xc0007ed600)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:119 +0x315
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.makeClientWorker
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:48 +0xe0
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: goroutine 284 [select]:
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*persistConn).roundTrip(0xc00bdab8c0, 0xd467d29620, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:2194 +0x75c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Transport).roundTrip(0xc000857d40, 0xc0007bc700, 0xd467d295f0, 0xd597d6642c, 0xd597d66420)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:481 +0xa1b
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Transport).RoundTrip(0xc000857d40, 0xc0007bc700, 0xc000857d40, 0xbf71c3620a1d249f, 0x2b6d248f5e)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/roundtrip.go:17 +0x35
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.send(0xc0007bd100, 0x33cfba0, 0xc000857d40, 0xbf71c3620a1d249f, 0x2b6d248f5e, 0x4e5a700, 0xc0000105f8, 0xbf71c3620a1d249f, 0x1, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:250 +0x461
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Client).send(0xc0005bbf20, 0xc0007bd100, 0xbf71c3620a1d249f, 0x2b6d248f5e, 0x4e5a700, 0xc0000105f8, 0x0, 0x1, 0x10)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:174 +0xfb
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Client).do(0xc0005bbf20, 0xc0007bd100, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:641 +0x279
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: net/http.(*Client).Do(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:509
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Connection).execHTTPRequest(0xc00023f080, 0xc0007bd100, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:826 +0x34c
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Connection).sendBulkRequest(...)
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/bulkapi.go:203
Dec 03 18:05:12 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Client).publishEvents(0xc00023f080, 0xc013f3a000, 0x21, 0x800, 0xc0007caba0, 0xc0007caba0, 0x0, 0xc00023f980, 0xc00023fa08)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:342 +0x341
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Client).Publish(0xc00023f080, 0x3448080, 0xc000894900, 0xc000802d20, 0xc00fe59f78)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:293 +0x43
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs.(*backoffClient).Publish(0xc0005bbf50, 0x3448080, 0xc000894900, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/backoff.go:60 +0x4b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*netClientWorker).run(0xc0007ed640)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:119 +0x315
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.makeClientWorker
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:48 +0xe0
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 286 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/monitoring/report/log.(*reporter).snapshotLoop(0xc0007ba510)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/monitoring/report/log/log.go:129 +0x3c5
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/monitoring/report/log.MakeReporter.func1(0xc0007ba510)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/monitoring/report/log/log.go:107 +0x53
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/monitoring/report/log.MakeReporter
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/monitoring/report/log/log.go:105 +0x159
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 790 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Input).stopWhenDone(0xc000983380)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:753 +0x11c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.NewInput
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:155 +0x936
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 779 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).writeLoop(0xc0009706c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1978 +0x113
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1360 +0xb0d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 946 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/x-pack/filebeat/input/netflow.(*netflowInput).recvRoutine(0xc0003ba870)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/input.go:237 +0x85
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/x-pack/filebeat/input/netflow.(*netflowInput).Run
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/input.go:167 +0x33d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 376 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/queue/memqueue.(*directEventLoop).run(0xc000402630)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/eventloop.go:100 +0x1d9
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/queue/memqueue.NewBroker.func1(0xc00023bf80, 0x33f0b40, 0xc000402630)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:175 +0x5d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/queue/memqueue.NewBroker
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:173 +0x374
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 377 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/queue/memqueue.(*ackLoop).run(0xc00015f6d0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/ackloop.go:60 +0x107
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/queue/memqueue.NewBroker.func2(0xc00023bf80, 0xc00015f6d0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:179 +0x57
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/queue/memqueue.NewBroker
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/queue/memqueue/broker.go:177 +0x3a3
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 378 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*eventConsumer).loop(0xc00015f7c0, 0x33f0b00, 0xc000743a60)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/consumer.go:190 +0x2c9
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.newEventConsumer
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/consumer.go:77 +0x155
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 379 [select, 1 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*retryer).loop(0xc00015f810)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/retry.go:139 +0x1df
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.newRetryer
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/retry.go:91 +0x13d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 380 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).roundTrip(0xc000800240, 0xc01bb27cb0, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:2194 +0x75c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*Transport).roundTrip(0xc00080cea0, 0xc0007bca00, 0xc01bb27a10, 0xd467ee36d0, 0xd467ee36c4)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:481 +0xa1b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*Transport).RoundTrip(0xc00080cea0, 0xc0007bca00, 0xc00080cea0, 0xbf71c35b762aa2fb, 0x254fdebf20)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/roundtrip.go:17 +0x35
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.send(0xc0007bc900, 0x33cfba0, 0xc00080cea0, 0xbf71c35b762aa2fb, 0x254fdebf20, 0x4e5a700, 0xc0007c0338, 0xbf71c35b762aa2fb, 0x1, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:250 +0x461
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*Client).send(0xc00083b320, 0xc0007bc900, 0xbf71c35b762aa2fb, 0x254fdebf20, 0x4e5a700, 0xc0007c0338, 0x0, 0x1, 0x10)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:174 +0xfb
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*Client).do(0xc00083b320, 0xc0007bc900, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:641 +0x279
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*Client).Do(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:509
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Connection).execHTTPRequest(0xc0004eb380, 0xc0007bc900, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/client.go:826 +0x34c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Connection).sendBulkRequest(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/bulkapi.go:203
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/elasticsearch.(*Connection).BulkWith(0xc0004eb380, 0xc0008ed8c0, 0x1e, 0x0, 0x0, 0x0, 0x0, 0xc00079fa60, 0x2, 0x2, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/elasticsearch/bulkapi.go:75 +0x17d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/monitoring/report/elasticsearch.(*publishClient).publishBulk(0xc000743680, 0xbf71c34c711442d2, 0x17528106c7, 0x4e5a700, 0xc01bb15620, 0xc01bb15650, 0x0, 0x0, 0x0, 0x0, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/monitoring/report/elasticsearch/client.go:233 +0x5cb
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/monitoring/report/elasticsearch.(*publishClient).Publish(0xc000743680, 0x3448080, 0xc02fb13280, 0xc00051a958, 0xc0004ebc38)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/monitoring/report/elasticsearch/client.go:140 +0x6b2
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs.(*failoverClient).Publish(0xc000743920, 0x3448080, 0xc02fb13280, 0xc0007c90e0, 0xc00079ff78)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/failover.go:99 +0x63
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs.(*backoffClient).Publish(0xc00083b5f0, 0x3448080, 0xc02fb13280, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/backoff.go:60 +0x4b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*netClientWorker).run(0xc00005af80)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:119 +0x315
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.makeClientWorker
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/output.go:48 +0xe0
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 381 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/service.HandleSignals.func1(0xc0008e39e0, 0xc000745ae0, 0xc00016b510, 0xc0008d6720)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/service/service.go:49 +0x41
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/service.HandleSignals
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/service/service.go:48 +0x115
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 789 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc001ecda10, 0xc0007c87e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 383 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*pipelineEventCB).worker(0xc00005b080)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/pipeline_ack.go:245 +0x1a1
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.newPipelineEventCB
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/pipeline_ack.go:196 +0x15b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 384 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/registrar.(*Registrar).Run(0xc00051b5c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/registrar/registrar.go:291 +0x161
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/registrar.(*Registrar).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/registrar/registrar.go:272 +0x13a
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 385 [select, 1 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc00015aa30)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 386 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc000852b40, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 387 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc00051b860)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc0007c6ef0, 0xc00051b860)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 388 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc00015b690)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 389 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc000858540, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 390 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc00015aea0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc0007c7480, 0xc00015aea0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 313 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9.(*SessionMap).CleanupLoop(0xc00015b6f8, 0x1a3185c5000, 0xc000796420)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9/session.go:182 +0xf5
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9.(*NetflowV9Protocol).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9/v9.go:62 +0x99
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 314 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9.(*SessionMap).CleanupLoop(0xc00015b758, 0x1a3185c5000, 0xc000796480)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9/session.go:182 +0xf5
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9.(*NetflowV9Protocol).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9/v9.go:62 +0x99
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 263 [IO wait]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d96b68, 0x72, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc000922098, 0x72, 0x2800, 0x2800, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).ReadFrom(0xc000922080, 0xc003ed4000, 0x2800, 0x2800, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:219 +0x194
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).readFrom(0xc000922080, 0xc003ed4000, 0x2800, 0x2800, 0x30d9518, 0xc0007f1e60, 0xc0007f1e50, 0x1308f6d, 0xc000922080)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:208 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*UDPConn).readFrom(0xc00018a0e0, 0xc003ed4000, 0x2800, 0x2800, 0xbf71c3970b978826, 0x5ccace3a13, 0x4e5a700, 0xe9)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*UDPConn).ReadFrom(0xc00018a0e0, 0xc003ed4000, 0x2800, 0x2800, 0x0, 0x0, 0x4e5a700, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/udpsock.go:121 +0x5e
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).run(0xc00005b240)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:106 +0x126
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start.func1(0xc00005b240)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:85 +0x57
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:83 +0x172
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 316 [IO wait]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d96d08, 0x72, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc000779198, 0x72, 0x200, 0x200, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).Read(0xc000779180, 0xc027ebec00, 0x200, 0x200, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).Read(0xc000779180, 0xc027ebec00, 0x200, 0x200, 0xc0005a1120, 0x128961a, 0x2d9f540)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:202 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*conn).Read(0xc0007ac268, 0xc027ebec00, 0x200, 0x200, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/net.go:177 +0x69
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/github.com/miekg/dns.(*Conn).Read(0xc033afb380, 0xc027ebec00, 0x200, 0x200, 0xc0007ac268, 0xc00005e501, 0xc00005e500)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/github.com/miekg/dns/client.go:261 +0x23f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/github.com/miekg/dns.(*Conn).ReadMsgHeader(0xc033afb380, 0x0, 0x0, 0x12a05e7d4, 0xc00fc86360, 0x12a05e7d4, 0xc0005a12c8)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/github.com/miekg/dns/client.go:225 +0xc7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/github.com/miekg/dns.(*Conn).ReadMsg(0xc033afb380, 0xbf71c352d59b052c, 0x1d44c04d26, 0x4e5a700)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/github.com/miekg/dns/client.go:187 +0x49
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/github.com/miekg/dns.(*Client).exchange(0xc00047a0e0, 0xc0005a1510, 0xc0004653c0, 0xf, 0x0, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/github.com/miekg/dns/client.go:173 +0x27a
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/vendor/github.com/miekg/dns.(*Client).Exchange(0xc00047a0e0, 0xc0005a1510, 0xc0004653c0, 0xf, 0x0, 0x12a089d40, 0x33cfaa0, 0xc0007b8b40)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/vendor/github.com/miekg/dns/client.go:129 +0x2bc
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/processors/dns.(*MiekgResolver).LookupPTR(0xc00057a910, 0xc013acae40, 0xe, 0x4e5a700, 0xc013acae40, 0xe)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/processors/dns/resolver.go:137 +0x235
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/processors/dns.PTRLookupCache.LookupPTR(0xc0005bb110, 0xc0005bb140, 0x0, 0x33cbf80, 0xc00057a910, 0xc0004653d0, 0xc0004653d8, 0xc013acae40, 0xe, 0x0, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/processors/dns/cache.go:195 +0x130
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/processors/dns.(*processor).processField(0xc000592bb0, 0xc000465360, 0xe, 0xc00033d380, 0x14, 0x12e6600, 0xc02ffed1c0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/processors/dns/dns.go:103 +0xc1
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/processors/dns.(*processor).Run(0xc000592bb0, 0xc02ffed1c0, 0xc02ffed1c0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/processors/dns/dns.go:84 +0x111
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/processing.(*group).Run(0xc00058ef60, 0xc02ffed1c0, 0xc02ffed1c0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/processing/processors.go:104 +0x94
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/processing.(*group).Run(0xc000858390, 0xc02ffed1c0, 0xed5788c59, 0x0, 0xc1dbb58f00)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/processing/processors.go:104 +0x94
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*client).publish(0xc0007a1080, 0x0, 0xed5788c59, 0x0, 0x0, 0xd597cea210, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/client.go:89 +0x5c3
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*client).Publish(0xc0007a1080, 0x0, 0xed5788c59, 0x0, 0x0, 0xd597cea210, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/client.go:68 +0x9c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.(*outlet).OnEvent(0xc000858540, 0x0, 0xed5788c59, 0x0, 0x0, 0xd597cea210, 0x0, 0x0, 0x0, 0x2887fa01)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/outlet.go:64 +0x80
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/harvester.(*Forwarder).Send(0xc0003f3ea0, 0x0, 0xed5788c59, 0x0, 0x0, 0xd597cea210, 0x0, 0x0, 0x0, 0x0, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/harvester/forwarder.go:50 +0x65
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/x-pack/filebeat/input/netflow.(*netflowInput).Publish(0xc00015fd10, 0xd597ce8000, 0x1d, 0x1d, 0xd467b40b10, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/input.go:138 +0x111
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/x-pack/filebeat/input/netflow.(*netflowInput).recvRoutine(0xc00015fd10)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/input.go:248 +0x32c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/x-pack/filebeat/input/netflow.(*netflowInput).Run
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/input.go:167 +0x33d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 950 [IO wait, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d96688, 0x72, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc0008aff98, 0x72, 0xdf00, 0xdf50, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).Read(0xc0008aff80, 0xc003d50000, 0xdf50, 0xdf50, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).Read(0xc0008aff80, 0xc003d50000, 0xdf50, 0xdf50, 0x203000, 0x0, 0x33)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:202 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*conn).Read(0xc00081c020, 0xc003d50000, 0xdf50, 0xdf50, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/net.go:177 +0x69
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*atLeastReader).Read(0xc00080a220, 0xc003d50000, 0xdf50, 0xdf50, 0x4, 0xa, 0xc00078d938)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:761 +0x60
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bytes.(*Buffer).ReadFrom(0xc00059f058, 0x33cb6c0, 0xc00080a220, 0x1289475, 0x2cf3640, 0x2f21b60)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bytes/buffer.go:207 +0xbd
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readFromUntil(0xc00059ee00, 0x33cfae0, 0xc00081c020, 0x5, 0xc00081c020, 0x203000)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:783 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecordOrCCS(0xc00059ee00, 0x30da000, 0xc00059ef38, 0xc00078db88)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:590 +0x125
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecord(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:558
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).Read(0xc00059ee00, 0xc001e7f000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:1236 +0x137
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).Read(0xc00072ca20, 0xc001e7f000, 0x1000, 0x1000, 0xc00078dc88, 0x1284665, 0xc000802360)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1526 +0x7b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).fill(0xc001e96000)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:100 +0x10f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).Peek(0xc001e96000, 0x1, 0x0, 0x0, 0x1, 0xc0007b5f00, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:138 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).readLoop(0xc00072ca20)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1679 +0x1a3
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1359 +0xae8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 951 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).writeLoop(0xc00072ca20)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1978 +0x113
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1360 +0xb0d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 891 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*TimeoutReader).Next(0xc00005b840, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc0008580a8, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/timeout.go:79 +0x12f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/multiline.(*Reader).readNext(0xc001e5a9c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc037c2ad20, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/multiline/multiline.go:166 +0x6e
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/multiline.(*Reader).Next(0xc001e5a9c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc002923d48, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/multiline/multiline.go:134 +0x6a
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*LimitReader).Next(0xc0008d6e20, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x257a01b, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/limit.go:40 +0x6b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Harvester).Run(0xc000286780, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/harvester.go:260 +0x2a1
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/harvester.(*Registry).Start.func1(0xc00005b900, 0x340dd80, 0xc000286780)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/harvester/registry.go:96 +0x73
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/harvester.(*Registry).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/harvester/registry.go:90 +0x128
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 847 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Log).wait(0xc00041ad20)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/log.go:195 +0xbc
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Log).Read(0xc00041ad20, 0xc0008d2000, 0x4000, 0x4000, 0x1, 0xffffffffffffffff, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/log.go:108 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*LineReader).advance(0xc00077a200, 0x8, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/line.go:133 +0x12f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*LineReader).Next(0xc00077a200, 0xc0007dd5d8, 0x128b369, 0xc000160fc0, 0x60, 0x60, 0x2f18940)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/line.go:78 +0x2f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.EncoderReader.Next(0xc00077a200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1284d2c, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/encode.go:53 +0x63
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*StripNewline).Next(0xc0005e7b00, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x13edd64, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/strip_newline.go:50 +0x6b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*TimeoutReader).Next.func1(0xc0007ec6c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/timeout.go:70 +0x48
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/reader/readfile.(*TimeoutReader).Next
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/timeout.go:68 +0x287
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 721 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Input).stopWhenDone(0xc0008c29c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:753 +0x11c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.NewInput
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:155 +0x936
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 786 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc001e62bb0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 845 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc0008091a0, 0xc0007b4b40)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 778 [IO wait, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d968f8, 0x72, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc000813e18, 0x72, 0x1a00, 0x1af9, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).Read(0xc000813e00, 0xc00089e000, 0x1af9, 0x1af9, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).Read(0xc000813e00, 0xc00089e000, 0x1af9, 0x1af9, 0x17b8d460eb9bfb, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:202 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*conn).Read(0xc00081c000, 0xc00089e000, 0x1af9, 0x1af9, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/net.go:177 +0x69
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*atLeastReader).Read(0xc0005e4000, 0xc00089e000, 0x1af9, 0x1af9, 0x4, 0xa, 0xc00094d938)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:761 +0x60
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bytes.(*Buffer).ReadFrom(0xc00059e958, 0x33cb6c0, 0xc0005e4000, 0x1289475, 0x2cf3640, 0x2f21b60)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bytes/buffer.go:207 +0xbd
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readFromUntil(0xc00059e700, 0x33cfae0, 0xc00081c000, 0x5, 0xc00081c000, 0x203000)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:783 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecordOrCCS(0xc00059e700, 0x30da000, 0xc00059e838, 0xc00094db88)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:590 +0x125
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecord(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:558
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).Read(0xc00059e700, 0xc00086c000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:1236 +0x137
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).Read(0xc0009706c0, 0xc00086c000, 0x1000, 0x1000, 0xc00094dc88, 0x1284665, 0xc0007b4a80)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1526 +0x7b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).fill(0xc00015a720)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:100 +0x10f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).Peek(0xc00015a720, 0x1, 0x0, 0x0, 0x1, 0xc000802600, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:138 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).readLoop(0xc0009706c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1679 +0x1a3
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1359 +0xae8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 846 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Harvester).Run.func2(0xc000286500, 0xc001edc7c0, 0x31)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/harvester.go:239 +0xca
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.(*Harvester).Run
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/harvester.go:232 +0x1d2
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 844 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc0008091a0, 0x340dbc0, 0xc001ebdce0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 780 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*TimeoutReader).Next(0xc0007ec6c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x33cb8e0, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/timeout.go:79 +0x12f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/multiline.(*Reader).readFirst(0xc0008f2410, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/multiline/multiline.go:139 +0x6b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/multiline.(*Reader).Next(0xc0008f2410, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x49c9651, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/multiline/multiline.go:134 +0x6a
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*LimitReader).Next(0xc0008b1a80, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x12afc2b, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/limit.go:40 +0x6b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Harvester).Run(0xc000286500, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/harvester.go:260 +0x2a1
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/harvester.(*Registry).Start.func1(0xc001ea51c0, 0x340dd80, 0xc000286500)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/harvester/registry.go:96 +0x73
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/harvester.(*Registry).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/harvester/registry.go:90 +0x128
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 840 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc0005906c0, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 839 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc000231ab0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 289 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).writeLoop(0xc000800240)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1978 +0x113
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1360 +0xb0d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 288 [IO wait]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d96c38, 0x72, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc000188418, 0x72, 0xc00, 0xc72, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).Read(0xc000188400, 0xc000452000, 0xc72, 0xc72, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).Read(0xc000188400, 0xc000452000, 0xc72, 0xc72, 0x203004, 0x0, 0x34)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:202 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*conn).Read(0xc00018a0d0, 0xc000452000, 0xc72, 0xc72, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/net.go:177 +0x69
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*atLeastReader).Read(0xc013f31b80, 0xc000452000, 0xc72, 0xc72, 0x3, 0xa, 0xc00078f938)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:761 +0x60
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bytes.(*Buffer).ReadFrom(0xc0004d8958, 0x33cb6c0, 0xc013f31b80, 0x1289475, 0x2cf3640, 0x2f21b60)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bytes/buffer.go:207 +0xbd
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readFromUntil(0xc0004d8700, 0x33cfae0, 0xc00018a0d0, 0x5, 0xc00018a0d0, 0x203007)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:783 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecordOrCCS(0xc0004d8700, 0x30da000, 0xc0004d8838, 0xc00078fb88)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:590 +0x125
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecord(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:558
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).Read(0xc0004d8700, 0xc0003ea000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:1236 +0x137
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).Read(0xc000800240, 0xc0003ea000, 0x1000, 0x1000, 0xc00078fc88, 0x1284665, 0xc000184240)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1526 +0x7b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).fill(0xc0008e2360)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:100 +0x10f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).Peek(0xc0008e2360, 0x1, 0x0, 0x0, 0x1, 0xc0000bff00, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:138 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).readLoop(0xc000800240)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1679 +0x1a3
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1359 +0xae8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 787 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc001ecd9e0, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 788 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc001ecda10, 0x340dbc0, 0xc001ecd9e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 513 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc0008f5d50)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 514 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc00083a6c0, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 515 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc0001609d0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 516 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc0008526c0, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 569 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc00015bf30)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 570 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc00084b5c0, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 272 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/beater.waitChannel.func1()
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/beater/signalwait.go:79 +0x33
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/beater.(*signalWait).Add.func1(0xc0009250f0, 0xc0001526b0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/beater/signalwait.go:52 +0x27
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/beater.(*signalWait).Add
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/beater/signalwait.go:51 +0x4c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 271 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/cfgfile.(*Reloader).Run(0xc00015b9e0, 0x33cba00, 0xc00015fd60)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/cfgfile/reload.go:228 +0x43e
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/crawler.(*Crawler).Start.func2(0xc000380510)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/crawler/crawler.go:102 +0x41
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/crawler.(*Crawler).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/crawler/crawler.go:101 +0x45f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 803 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).writeLoop(0xc001eec240)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1978 +0x113
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1360 +0xb0d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 802 [IO wait, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d969c8, 0x72, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc000189d98, 0x72, 0xc00, 0xc72, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).Read(0xc000189d80, 0xc000452c80, 0xc72, 0xc72, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).Read(0xc000189d80, 0xc000452c80, 0xc72, 0xc72, 0x203000, 0x12a0475, 0xc0008d6320)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:202 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*conn).Read(0xc00018a070, 0xc000452c80, 0xc72, 0xc72, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/net.go:177 +0x69
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*atLeastReader).Read(0xc0008d6320, 0xc000452c80, 0xc72, 0xc72, 0x4e63020, 0x7f56b8dfb908, 0xc000789938)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:761 +0x60
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bytes.(*Buffer).ReadFrom(0xc000324cd8, 0x33cb6c0, 0xc0008d6320, 0x1289475, 0x2cf3640, 0x2f21b60)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bytes/buffer.go:207 +0xbd
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readFromUntil(0xc000324a80, 0x33cfae0, 0xc00018a070, 0x5, 0xc00018a070, 0x203000)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:783 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecordOrCCS(0xc000324a80, 0x30da000, 0xc000324bb8, 0xc000789b88)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:590 +0x125
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecord(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:558
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).Read(0xc000324a80, 0xc003ccc000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:1236 +0x137
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).Read(0xc001eec240, 0xc003ccc000, 0x1000, 0x1000, 0xc000789c88, 0x1284665, 0xc0007b5200)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1526 +0x7b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).fill(0xc0007a6e40)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:100 +0x10f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).Peek(0xc0007a6e40, 0x1, 0x0, 0x0, 0x1, 0xc0001d0600, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:138 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).readLoop(0xc001eec240)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1679 +0x1a3
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1359 +0xae8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 270 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Input).stopWhenDone(0xc0008c21a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:753 +0x11c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.NewInput
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:155 +0x936
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 617 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc0008055d0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 618 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc000910630, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 619 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc000910660, 0x340dbc0, 0xc000910630)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 620 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc000910660, 0xc0007c87e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 621 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Input).stopWhenDone(0xc0009824e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:753 +0x11c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.NewInput
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:155 +0x936
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 622 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc0007ca910)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 623 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc0007eef90, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 624 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc0007eefc0, 0x340dbc0, 0xc0007eef90)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 625 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc0007eefc0, 0xc0007c87e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 626 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Input).stopWhenDone(0xc000982820)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:753 +0x11c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.NewInput
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:155 +0x936
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1404 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.setRequestCancel.func3(0x0, 0xd467d295f0, 0xc0003ba320, 0xd597d6642c, 0xc0007c8120)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:321 +0xcf
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.setRequestCancel
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:320 +0x251
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 155 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).writeLoop(0xc000909680)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1978 +0x113
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1360 +0xb0d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 269 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc0007834d0, 0xc0007c87e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 666 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc0007b7a50)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 667 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc0005e7020, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 267 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc0007834a0, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 268 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc0007834d0, 0x340dbc0, 0xc0007834a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 266 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc00051abb0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 265 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Input).stopWhenDone(0xc000983040)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:753 +0x11c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.NewInput
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:155 +0x936
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 867 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc000230840)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc000810060, 0xc000230840)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 715 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc00088db70)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 716 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc0008977d0, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 717 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc000897800, 0x340dbc0, 0xc0008977d0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 718 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc000897800, 0xc0007c87e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 719 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/monitoring/report/elasticsearch.(*reporter).snapshotLoop(0xc00047a540, 0x2fdf81c, 0x5, 0x2fdf81c, 0x5, 0xdf8475800, 0xc013e759c0, 0x16)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/monitoring/report/elasticsearch/elasticsearch.go:282 +0x2f4
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/monitoring/report/elasticsearch.(*reporter).initLoop
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/monitoring/report/elasticsearch/elasticsearch.go:265 +0x473
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 720 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/monitoring/report/elasticsearch.(*reporter).snapshotLoop(0xc00047a540, 0x2fdf826, 0x5, 0x2fe2c86, 0x7, 0x2540be400, 0xc013e759c0, 0x16)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/monitoring/report/elasticsearch/elasticsearch.go:282 +0x2f4
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/monitoring/report/elasticsearch.(*reporter).initLoop
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/monitoring/report/elasticsearch/elasticsearch.go:267 +0x4e9
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 774 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc001eb8c70)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 775 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc001ebdce0, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 776 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc001ebdd10, 0x340dbc0, 0xc001ebdce0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 777 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc001ebdd10, 0xc0007c87e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 781 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc000231620)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc0004658b0, 0xc000231620)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 160 [IO wait]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d97048, 0x72, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc00fb75698, 0x72, 0xf900, 0xf950, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).Read(0xc00fb75680, 0xc013d00000, 0xf950, 0xf950, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).Read(0xc00fb75680, 0xc013d00000, 0xf950, 0xf950, 0x203000, 0x0, 0xf943)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:202 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*conn).Read(0xc00081c060, 0xc013d00000, 0xf950, 0xf950, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/net.go:177 +0x69
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*atLeastReader).Read(0xc000781200, 0xc013d00000, 0xf950, 0xf950, 0x40, 0x17ab, 0xc00008d8d8)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:761 +0x60
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bytes.(*Buffer).ReadFrom(0xc00059fad8, 0x33cb6c0, 0xc000781200, 0x1289475, 0x2cf3640, 0x2f21b60)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bytes/buffer.go:207 +0xbd
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readFromUntil(0xc00059f880, 0x33cfae0, 0xc00081c060, 0x5, 0xc00081c060, 0x40)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:783 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecordOrCCS(0xc00059f880, 0x30da000, 0xc00059f9b8, 0xc00008db38)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:590 +0x125
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecord(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:558
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).Read(0xc00059f880, 0xc00fd56000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:1236 +0x137
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/outputs/transport.(*statsConn).Read(0xc00bd98ac0, 0xc00fd56000, 0x1000, 0x1000, 0xc00008dc50, 0x1284b8c, 0xc00fc01860)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/outputs/transport/stats.go:44 +0x55
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).Read(0xc00fcc57a0, 0xc00fd56000, 0x1000, 0x1000, 0xc00008dc88, 0x1284665, 0xc00fc01860)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1526 +0x7b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).fill(0xc0007a3620)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:100 +0x10f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).Peek(0xc0007a3620, 0x1, 0x0, 0x0, 0x1, 0xc000797500, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:138 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).readLoop(0xc00fcc57a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1679 +0x1a3
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1359 +0xae8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 161 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).writeLoop(0xc00fcc57a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1978 +0x113
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1360 +0xb0d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 794 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc001eb83c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc00033e0a0, 0xc001eb83c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com systemd[1]: filebeat.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 795 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc001e62360)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc00033e1f0, 0xc001e62360)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 800 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9.(*SessionMap).CleanupLoop(0xc000231b18, 0x1a3185c5000, 0xc0000bf020)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9/session.go:182 +0xf5
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9.(*NetflowV9Protocol).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9/v9.go:62 +0x99
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 868 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc0002313e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc000810080, 0xc0002313e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 801 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9.(*SessionMap).CleanupLoop(0xc000231b78, 0x1a3185c5000, 0xc0000bf080)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9/session.go:182 +0xf5
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9.(*NetflowV9Protocol).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/x-pack/filebeat/input/netflow/decoder/v9/v9.go:62 +0x99
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 849 [IO wait, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d96828, 0x72, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com systemd[1]: Unit filebeat.service entered failed state.
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc000812118, 0x72, 0x2800, 0x2800, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).ReadFrom(0xc000812100, 0xc000872800, 0x2800, 0x2800, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:219 +0x194
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).readFrom(0xc000812100, 0xc000872800, 0x2800, 0x2800, 0x30d9518, 0xc001ef1e60, 0xc001ef1e50, 0x1308f6d, 0xc000812100)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:208 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*UDPConn).readFrom(0xc0007ac028, 0xc000872800, 0x2800, 0x2800, 0xbf71c37e75734622, 0x45e7cdd9f6, 0x4e5a700, 0xe9)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*UDPConn).ReadFrom(0xc0007ac028, 0xc000872800, 0x2800, 0x2800, 0x0, 0x0, 0x4e5a700, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/udpsock.go:121 +0x5e
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).run(0xc0007e7040)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:106 +0x126
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start.func1(0xc0007e7040)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:85 +0x57
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:83 +0x172
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 931 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc000230d90)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 932 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc00058f830, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com systemd[1]: filebeat.service failed.
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 933 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc00058f860, 0x340dbc0, 0xc00058f830)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 934 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc00058f860, 0xc0007c87e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 935 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Input).stopWhenDone(0xc0000dc680)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:753 +0x11c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.NewInput
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:155 +0x936
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 936 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc000231990)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 937 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc0005e6c90, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 938 [chan receive, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc0005e6cc0, 0x340dbc0, 0xc0005e6c90)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 939 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc0005e6cc0, 0xc0007c87e0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 940 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Input).stopWhenDone(0xc0000dc9c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:753 +0x11c
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.NewInput
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/input.go:155 +0x936
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1570 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.setRequestCancel.func3(0x0, 0xc01bb27a10, 0xc000050aa0, 0xd467ee36d0, 0xc0001d0300)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:321 +0xcf
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.setRequestCancel
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:320 +0x251
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 154 [IO wait, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d96758, 0x72, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc00077ae98, 0x72, 0x2000, 0x2011, 0xffffffffffffffff)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).Read(0xc00077ae80, 0xc00020a500, 0x2011, 0x2011, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).Read(0xc00077ae80, 0xc00020a500, 0x2011, 0x2011, 0x203000, 0x0, 0x32)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:202 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*conn).Read(0xc00018a078, 0xc00020a500, 0x2011, 0x2011, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/net.go:177 +0x69
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*atLeastReader).Read(0xc0008b0100, 0xc00020a500, 0x2011, 0x2011, 0x4e63020, 0x7f56b8d97fd0, 0xc000941938)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:761 +0x60
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bytes.(*Buffer).ReadFrom(0xc000648958, 0x33cb6c0, 0xc0008b0100, 0x1289475, 0x2cf3640, 0x2f21b60)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bytes/buffer.go:207 +0xbd
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readFromUntil(0xc000648700, 0x33cfae0, 0xc00018a078, 0x5, 0xc00018a078, 0x203000)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:783 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecordOrCCS(0xc000648700, 0x30da000, 0xc000648838, 0xc000941b88)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:590 +0x125
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).readRecord(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:558
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: crypto/tls.(*Conn).Read(0xc000648700, 0xc000748000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/crypto/tls/conn.go:1236 +0x137
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).Read(0xc000909680, 0xc000748000, 0x1000, 0x1000, 0xc000941c88, 0x1284665, 0xc0007b4de0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1526 +0x7b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).fill(0xc000451380)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:100 +0x10f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: bufio.(*Reader).Peek(0xc000451380, 0x1, 0x0, 0x0, 0x1, 0xc0007c9c00, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/bufio/bufio.go:138 +0x4f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).readLoop(0xc000909680)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1679 +0x1a3
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1359 +0xae8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 493 [running]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine running on other thread; stack unavailable
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:83 +0x172
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1154 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.(*persistConn).writeLoop(0xc00bdab8c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1978 +0x113
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.(*Transport).dialConn
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/transport.go:1360 +0xb0d
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 952 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc0007b65a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc0007929b0, 0xc0007b65a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 896 [IO wait, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d96418, 0x72, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc0008ae098, 0x72, 0x2800, 0x2800, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).ReadFrom(0xc0008ae080, 0xc003cce000, 0x2800, 0x2800, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:219 +0x194
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).readFrom(0xc0008ae080, 0xc003cce000, 0x2800, 0x2800, 0x30d9518, 0xc000633e60, 0xc000633e50, 0x1308f6d, 0xc0008ae080)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:208 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*UDPConn).readFrom(0xc0007c0070, 0xc003cce000, 0x2800, 0x2800, 0xbf71c37e8090aec9, 0x45ee860cc2, 0x4e5a700, 0xe9)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*UDPConn).ReadFrom(0xc0007c0070, 0xc003cce000, 0x2800, 0x2800, 0x0, 0x0, 0x4e5a700, 0xc000633fa8, 0xc000633fa8)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/udpsock.go:121 +0x5e
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).run(0xc0007ed7c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:106 +0x126
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start.func1(0xc0007ed7c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:85 +0x57
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:83 +0x172
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1556 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.setRequestCancel.func3(0x0, 0xd467c455f0, 0xc0007a4000, 0xc01bd2c59c, 0xc00bd91140)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:321 +0xcf
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.setRequestCancel
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:320 +0x251
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 953 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc0007b6de0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc000792a00, 0xc0007b6de0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1124 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Log).wait(0xc00023aa10)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/log.go:195 +0xbc
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Log).Read(0xc00023aa10, 0xc02ff52000, 0x4000, 0x4000, 0x1, 0xffffffffffffffff, 0x4b3)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/log.go:108 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*LineReader).advance(0xc000778600, 0x4b3, 0x500)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/line.go:133 +0x12f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*LineReader).Next(0xc000778600, 0x4b3, 0x500, 0x2fdd3ae, 0x3, 0xc03bb10500, 0x4b3)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/line.go:78 +0x2f
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.EncoderReader.Next(0xc000778600, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/encode.go:53 +0x63
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*StripNewline).Next(0xc00084a870, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/strip_newline.go:50 +0x6b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/reader/readfile.(*TimeoutReader).Next.func1(0xc00005b840)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/timeout.go:70 +0x48
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/reader/readfile.(*TimeoutReader).Next
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/reader/readfile/timeout.go:68 +0x287
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 954 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Run(0xc0007b75c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:147 +0x145
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input.(*Runner).Start.func1(0xc000792b70, 0xc0007b75c0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:132 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input.(*Runner).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/input.go:125 +0x16b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1123 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/input/log.(*Harvester).Run.func2(0xc000286780, 0xc00016e8a0, 0x11)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/harvester.go:239 +0xca
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/input/log.(*Harvester).Run
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/input/log/harvester.go:232 +0x1d2
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1122 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dc00, 0xc000858090, 0xc0000bf440)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1058 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc0007b6b50)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1059 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc000777770, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1060 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc0007b7330)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1061 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc000972de0, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1114 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).ackLoop(0xc000161810)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:141 +0xff
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/libbeat/publisher/pipeline.(*gapCountACK).init
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/libbeat/publisher/pipeline/acker.go:129 +0x182
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1115 [select, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.CloseOnSignal.func1(0x340dbc0, 0xc0008c8150, 0xc0007c94a0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:112 +0xb7
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.CloseOnSignal
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:111 +0x76
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 955 [IO wait, 2 minutes]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.runtime_pollWait(0x7f56b8d964e8, 0x72, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/runtime/netpoll.go:182 +0x56
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).wait(0xc0007a0298, 0x72, 0x2800, 0x2800, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*pollDesc).waitRead(...)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_poll_runtime.go:92
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: internal/poll.(*FD).ReadFrom(0xc0007a0280, 0xc003cb8000, 0x2800, 0x2800, 0x0, 0x0, 0x0, 0x0, 0x0)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/internal/poll/fd_unix.go:219 +0x194
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*netFD).readFrom(0xc0007a0280, 0xc003cb8000, 0x2800, 0x2800, 0x30d9518, 0xc001ef9e60, 0xc001ef9e50, 0x1308f6d, 0xc0007a0280)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/fd_unix.go:208 +0x5b
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*UDPConn).readFrom(0xc00081c0c0, 0xc003cb8000, 0x2800, 0x2800, 0xbf71c37e808b2425, 0x45ee808206, 0x4e5a700, 0xe9)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/udpsock_posix.go:47 +0x6a
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net.(*UDPConn).ReadFrom(0xc00081c0c0, 0xc003cb8000, 0x2800, 0x2800, 0x0, 0x0, 0x4e5a700, 0xc001ef9fa8, 0x13edd64)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/udpsock.go:121 +0x5e
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).run(0xc0005b6480)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:106 +0x126
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start.func1(0xc0005b6480)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:85 +0x57
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/inputsource/udp.(*Server).Start
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/inputsource/udp/server.go:83 +0x172
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 945 [chan receive]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: github.com/elastic/beats/filebeat/channel.SubOutlet.func1(0xc000858090, 0x340dbc0, 0xc0005e6c90)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:44 +0x150
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by github.com/elastic/beats/filebeat/channel.SubOutlet
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /go/src/github.com/elastic/beats/filebeat/channel/util.go:43 +0xf8
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: goroutine 1405 [select]:
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: net/http.setRequestCancel.func3(0x0, 0xc01bdb2c90, 0xc0003ba550, 0xd597d66d8c, 0xc0007c8420)
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:321 +0xcf
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: created by net/http.setRequestCancel
Dec 03 18:05:13 my.file.beat.com filebeat[30053]: /usr/local/go/src/net/http/client.go:320 +0x251
Dec 03 18:05:13 my.file.beat.com systemd[1]: filebeat.service holdoff time over, scheduling restart.
Dec 03 18:05:13 my.file.beat.com systemd[1]: Stopped Filebeat sends log files to Logstash or directly to Elasticsearch..
Dec 03 18:05:13 my.file.beat.com systemd[1]: Started Filebeat sends log files to Logstash or directly to Elasticsearch..
```

So I did remove the `add_fields` for the additional `destination.geo` and it worked again. At the time where filebeat was consuming that large amount of memory, I saw in htop that RAM and swap were 100% used, within 10-15 seconds, after that filebeat was restarted, the RAM was emptied and it started again. Since then it looks like this

![10 11 107 89_5601_app_monitoring (7)](https://user-images.githubusercontent.com/12175559/70079133-65186800-1604-11ea-8260-b92ad4d7c927.png)

So to conclude:
- at some critical amount of `add_fields` operations, filebeat may break?
- is this "memory leak" only in conjunction with netflow?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Filebeat netflow GC "collecting" 402GB of RAM

Description

Contributor guide

Assessment

Get new issues in your inbox