elastic / elastic/ecs-logging-java

Docker configuration

Abierto
#43 5 comentarios 0 reacciones 0 asignados Ver en GitHub
agent-java community
Lenguaje dominante
Java
Estrellas
148
Forks
82
Merge medio
2 d 17 h
PR fusionados (30 d)
3

Descripción

Hi,

I am having a hard time working out how to use it in conjunction with docker. I have a Java application that logs everything in JSON to a file inside the docker container. I have created a `ln -sf /dev/stdout /opt/application.json` symlink, so everything will be output to stdout. Of course, looking into the -json.log file created by Docker in the `/var/lib/docker/container/containerid/*-json.log` it looks something like this:

```json
{"log":"{\"@timestamp\":\"2019-11-03T12:33:45.658Z\", \"log.level\": \"INFO\", \"message\":\"Login pkahr (philipp.kahr)\", \"service.name\":\"myservice\",\"process.thread.name\":\"http-bio-8080-exec-8\",\"log.logger\":\"com.myservice.server.core.utils.user.ThisDirectoryServiceImpl\",\"log.origin\":{\"file.name\":\"ThisDirectoryServiceImpl.java\",\"function\":\"loadUserByUsername\",\"file.line\":622}}\n","stream":"stdout","time":"2019-11-03T12:33:45.658546603Z"}
```

My filebeat config looks like this:

```yaml
filebeat.inputs:
- type: container
paths:
- '/var/lib/docker/containers/*/*.log'
json.keys_under_root: true
json.overwrite_keys: true
json.message_key: log
multiline.pattern: '^{'
multiline.negate: true
multiline.match: after
```

I am using the message decoder like suggested in the README
```yaml
- decode_json_fields:
fields: message
target: ""
overwrite_keys: true
# flattens the array to a single string
- script:
when:
has_fields: ['error.stack_trace']
lang: javascript
id: my_filter
source: >
function process(event) {
event.Put("error.stack_trace", event.Get("error.stack_trace").join("\n"));
}
```

however, I can see a couple of errors in `journalctl -u filebeat -f`

```
filebeat[29865]: 2019-11-03T13:56:09.252+0100 ERROR readjson/json.go:52 Error decoding JSON: json: cannot unmarshal number into Go value of type map[string]interface {}
filebeat[29865]: 2019-11-03T13:56:21.269+0100 ERROR readjson/json.go:52 Error decoding JSON: invalid character '-' in numeric literal
filebeat[29865]: 2019-11-03T13:56:21.269+0100 ERROR readjson/json.go:52 Error decoding JSON: invalid character 'E' looking for beginning of value
filebeat[29865]: 2019-11-03T13:56:21.268+0100 ERROR readjson/json.go:52 Error decoding JSON: unexpected EOF

filebeat[29865]: 2019-11-03T13:56:17.267+0100 WARN elasticsearch/client.go:535 Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0x120f69c0, ext:63708382571, loc:(*time.Location)(nil)}, Meta:common.MapStr(nil), Fields:common.MapStr{"agent":common.MapStr{"ephemeral_id":"f3bb8324-dda5-4dbe-9f1a-6fc86820c37d", "hostname":"CentOS-76-64-minimal", "id":"d8cb4370-3918-40a8-88c5-80a992868234", "type":"filebeat", "version":"7.4.2"}, "ecs":common.MapStr{"version":"1.1.0"}, "host":common.MapStr{"architecture":"x86_64", "containerized":false, "hostname":"CentOS-76-64-minimal", "id":"7d91e31c18374a518bb44ad58646dfeb", "name":"CentOS-76-64-minimal", "os":common.MapStr{"codename":"Core", "family":"redhat", "kernel":"3.10.0-1062.4.1.el7.x86_64", "name":"CentOS Linux", "platform":"centos", "version":"7 (Core)"}}, "input":common.MapStr{"type":"container"}, "log":"", "log.level":"INFO", "log.logger":"com.myservice.server.core.utils.user.ServiceDirectoryServiceImpl", "log.origin":common.MapStr{"file.line":622, "file.name":"ServiceDirectoryServiceImpl.java", "function":"loadUserByUsername"}, "message":"Login pkahr (philipp.kahr)", "process.thread.name":"http-bio-8080-exec-2", "service.name":"myservice", "stream":"stdout", "tags":[]string{"testing"}}, Private:file.State{Id:"", Finished:false, Fileinfo:(*os.fileStat)(0xc00022cc30), Source:"/var/lib/docker/containers/b35f79cc3aa44c98709c3a05dfee5fcc13c92d18b0cff35d821af9820e52c398/b35f79cc3aa44c98709c3a05dfee5fcc13c92d18b0cff35d821af9820e52c398-json.log", Offset:656232, Timestamp:time.Time{wall:0xbf67d03f87d679e9, ext:12500845494, loc:(*time.Location)(0x4de6580)}, TTL:-1, Type:"container", Meta:map[string]string(nil), FileStateOS:file.StateOS{Inode:0x18802e6, Device:0x902}}, TimeSeries:false}, Flags:0x1} (status=400): {"type":"mapper_parsing_exception","reason":"object mapping for [log] tried to parse field [log] as object, but found a concrete value"}
```

Any idea on how to use the ECS logging in combination with docker? I would love to have it, so I can jump from the APM to the `container logs`.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.