elastic / elastic/apm

Send logs through APM Agents

Open
#693 0 comments 6 reactions 0 assignees View on GitHub
apm-agents meta
Dominant language
Gherkin
Stars
427
Forks
125
PR merge metrics
No merged PRs in 30d

Description

# Description
Onboarding application logs has improved with ECS logging libraries but is still not as easy as it could be.

For users that are already using APM Agents, the most pleasant experience would be to just enable a config setting (such as capture_logs=true) and be done with it.

For agents that currently support the [ecs_log_reformatting](https://github.com/elastic/apm/blob/main/specs/agents/log-onboarding.md#log_ecs_reformatting-configuration) options, this would be a relatively simple addition. Similar to the just mentioned option, agents introduce a capture_logs (naming tbd) option.

Instead of the option modifying the log output, it captures the log events from the log library, serializes it using ecs-logging-, and puts the resulting string or byte array into the reporting queue so that it ca be sent to APM Server using the intake v2 protocol.

Agents that don't support auto-instrumentation (for example, the Go agent) can offer a log appender that can be configured with a log formatter from ecs-logging-go. The appender will then also just format the log event for the respective logging library and add the formatted log to the agent's internal reporter queue.

Central config for `capture_logs` is not in scope, however we should evaluate the feasibility across agents.

- [ ] Add support for fields from the ECS logging spec (including dotted and nested notations) https://github.com/elastic/apm-server/issues/9100
- [ ] Manual end-to-end testing
- [ ] Add telemetry to track the number of services that have logs

## Spec Issue
- [ ] https://github.com/elastic/apm/issues/694 Milestone

## Agent Issues
- [x] https://github.com/elastic/apm-agent-java/issues/2811 Milestone
- [ ] https://github.com/elastic/apm-agent-dotnet/issues/1844 Milestone
- [ ] https://github.com/elastic/apm-agent-nodejs/issues/2891 Milestone
- [ ] https://github.com/elastic/apm-agent-python/issues/1650 Milestone
- [ ] https://github.com/elastic/apm-agent-go/issues/1329 Milestone
- [ ] https://github.com/elastic/apm-agent-php/issues/782 Milestone
- [ ] https://github.com/elastic/apm-agent-ruby/issues/1312 Milestone

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.