could not pack/validate JSON response
- Dominant language
- C
- Stars
- 8.1k
- Forks
- 2k
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 71
Description
## Bug Report
**Describe the bug**
I'm running fluent-bit inside kubernetes to pass pod log entries to Elastisearch.
I can see majority of the logs coming in, however I've noticed the following errors from fluent-bit pods running on each kubernetes node.
How can I identify which pod is causing the error?
How can I resolve the issue below?
```
logs/fluent-bit-zjbxx[fluent-bit]: [2019/10/23 07:16:53] [error] [out_es] could not pack/validate JSON response
logs/fluent-bit-nqdwb[fluent-bit]: [2019/10/23 07:17:00] [error] [out_es] could not pack/validate JSON response
```
**Environment**
Kubernetes 1.13
fluent-bit helm chart
fluent-bit 1.2.2
**Configmap** from kubernetes
```apiVersion: v1
data:
fluent-bit-filter.conf: |-
[FILTER]
Name kubernetes
Match kube.*
Kube_Tag_Prefix kube.var.log.containers.
Kube_URL https://kubernetes.default.svc:443
Kube_CA_File /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
Kube_Token_File /var/run/secrets/kubernetes.io/serviceaccount/token
Merge_Log On
K8S-Logging.Parser On
K8S-Logging.Exclude On
Merge_Log_Trim On
fluent-bit-input.conf: "[INPUT]\n Name tail\n Path /var/log/containers/*.log\n
\ Parser docker\n Tag kube.*\n Refresh_Interval
5\n Mem_Buf_Limit 5MB\n Skip_Long_Lines On\n "
fluent-bit-output.conf: "\n[OUTPUT]\n Name es\n Match *\n Host XXXX\n
\ Port 9200\n Logstash_Format On\n Retry_Limit False\n Type flb_type\n
\ Time_Key @timestamp\n Replace_Dots On\n Logstash_Prefix kubernetes_cluster\n\n
\ HTTP_User admin\n HTTP_Passwd XXXX\n\n tls on\n tls.verify
off\n tls.debug 1\n tls.ca_file /secure/es-tls-ca.crt\n\n "
fluent-bit-service.conf: |-
[SERVICE]
Flush 1
Daemon Off
Log_Level info
Parsers_File parsers.conf
Parsers_File parsers_custom.conf
fluent-bit.conf: |-
@INCLUDE fluent-bit-service.conf
@INCLUDE fluent-bit-input.conf
@INCLUDE fluent-bit-filter.conf
@INCLUDE fluent-bit-output.conf
kind: ConfigMap
metadata:
annotations:
flux.weave.works/antecedent: logs:helmrelease/fluent-bit
creationTimestamp: "2019-05-30T08:05:58Z"
labels:
app: fluent-bit
chart: fluent-bit-2.7.3
heritage: Tiller
release: fluent-bit
name: fluent-bit-config
namespace: logs
resourceVersion: "99683776"
selfLink: /api/v1/namespaces/logs/configmaps/fluent-bit-config
uid: c1c81cd4-82b1-11e9-bace-06a20d1eb048
```
Contributor guide
Assessment
This issue has not been assessed yet.