elastic / elastic/ecs-logging-nodejs
ecs-pino-format does not integrate with elastic-apm-node when it is wrapped inside a library
- Dominant language
- JavaScript
- Stars
- 69
- Forks
- 45
- PR merge metrics
- No merged PRs in 30d
Description
Hi. We have an internal NPM package for logging, which wraps Pino and ecs-pino-format to format logs. We are using this library in multiple services. We also use elastic-apm-node in these services.
The problem is that the automatic integration of elastic-apm-node doesn't work when ecs-pino-format is wrapped inside this library, but it works when I directly import Pino and log from the service. `trace.id` and `event.dataset` fields are not added to the logs that come from the library.
What I'm using:
Typescript in both the service and the library.
"@elastic/ecs-pino-format": "^1.1.2"
"elastic-apm-node": "^3.9.0"
Example to explain what I mean:
I created 2 folders, one to simulate the service and the other to simulate the library. I `npm link`ed the library with the service.
pino-format-issue-service:

pino-format-issue-lib:

Log output:
```
{"log.level":"info","@timestamp":"2022-10-25T09:16:28.411Z","process":{"pid":71004},"host":{"hostname":""},"ecs":{"version":"1.6.0"},"service":{"name":"test-service"},"event":{"dataset":"test-service.log"},"message":"info from pino"}
{"log.level":"info","@timestamp":"2022-10-25T09:16:28.412Z","process":{"pid":71004},"host":{"hostname":""},"ecs":{"version":"1.6.0"},"message":"info from lib"}
```
As you can see, `event.dataset` field is added to the first log (directly logging from `pino`), but it's not there in the second log.
What I've tried:
1. Importing `elastic-apm-node/start` in both the service and the library.
This adds `trace.id` to the logs from library as well, but the ids from the service and library do not match as the agent is imported twice. This defeats the purpose as I will not be able to jump from APM to the log.
2. Importing the agent only inside the library, and removing it from the service.
I would be fine with doing this, but it seems that incoming and outgoing requests are not traced automatically when I do this, so there is no trace on APM.
3. Sending the request and response to the library using a middleware, and adding `trace.id` manually to the logs.
This honestly seems too complicated, and even led to a memory leak in the code that maintains request and response using `async_hooks`.
Is there a simpler way to achieve this that I'm missing? I really want to integrate APM and Logs for better debugging, and I also don't want to remove this library and drop to using bare Pino.
Thanks.
Contributor guide
Assessment
This issue has not been assessed yet.