elastic / elastic/apm-agent-nodejs

Transactions and spans confused about (custom) transaction parent

Open
#1,720 2 comments 0 reactions 0 assignees View on GitHub
agent-nodejs bug
Dominant language
JavaScript
Stars
594
Forks
244
Avg merge
1d 8h
Merged PRs (30d)
16

Description

**Describe the bug**

I am attempting to integrate the APM agent into a multi-service stack that communicates through a message bus, making use of the distributed tracing feature. I am running into several issues, but the root cause seems to be that child transactions and spans regularly fail to determine what their parent is.

In its most basic form, child spans (generated by instrumentation) often end up on the wrong parent when transactions are running concurrently. This happens with with `asyncHooks` enabled and disabled.

For example, a PostgreSQL query through Sequalize will end up on the wrong parent, while another sequelize query called in through an async function does not show up at all. This is demonstrated in the (psuedo)code below.

Other issues this leads to are:

- Custom transactions started in services called by a parent service (through the message bus) do not always show up on the correct parent transaction
- HTTP requests made through superagent do not show up. This is similar to https://github.com/elastic/apm-agent-nodejs/issues/897, however disabling `asyncHooks` does not help.

This issue may be related to https://github.com/elastic/apm-agent-nodejs/issues/1690

**To Reproduce**

```
const transaction = apm.startTransaction('GET CUSTOMER', 'test');
apm.setCustomContext({
...
});

... (no async code)

const customer = await Customer.findOne({
...
});

const foobar = await getFoobarFunction(...);

transaction.end('successful');
```

**Expected behavior**

Expected is that child transactions/spans show up under the correct parent.

**Environment (please complete the following information)**

- OS: Linux (Ubuntu 18.10)
- Node.js version: 10
- APM Server version: 7.6.2
- Agent version: 3.5.0

**How are you starting the agent? (please tick one of the boxes)**

- [x] Calling `agent.start()` directly (e.g. `require('elastic-apm-node').start(...)`)
- [ ] Requiring `elastic-apm-node/start` from within the source code
- [ ] Starting node with `-r elastic-apm-node/start`

**Additional context**

This screenshot is a typical result of the sample code above. It shows the following issues:

- Database queries from several other transactions have ended up under this transaction as spans. Many of the transactions of the same name are missing database spans.
- The database query called in the async function (`getFoobarFunction` in the sample above) does not show up in any of the transactions

![Screenshot from 2020-04-21 14-52-30](https://user-images.githubusercontent.com/330675/79870297-0f8dfd80-83e3-11ea-94a0-fad509c1216b.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.