elastic / elastic/apm-agent-nodejs

Spans not shown in the UI when their parent is missing

Open
#2,160 8 comments 0 reactions 0 assignees View on GitHub
agent-nodejs
Dominant language
JavaScript
Stars
594
Forks
244
Avg merge
1d 8h
Merged PRs (30d)
16

Description

First, note that I'm not sure if this is a bug in the UI or an issue with the approach suggest in the node.js docs, which is why I'm opening this issue here.

**Describe the bug**

We use a span filter to only store spans above a certain duration. The logic behind that is rather simple: most of the spans are very short and of little value. Storing only spans above a certain duration saves a lot of space in the ES. This approach has been suggested in the node.js agent repo (elastic/apm-agent-nodejs#1701) and is now included in the [performance tuning tips](https://www.elastic.co/guide/en/apm/agent/nodejs/current/performance-tuning.html#performance-transaction-max-spans).

We found, however, that often even long spans are missing from the APM UI (i.e., there would be a transaction but without spans), even though the spans are actually stored and possible to find via Discover.

I believe this happens when a short span is dropped by the filter, while it was set as a parent of some much longer span. The UI doesn't show the long span, because it can't find its parent (the following message suggests that):

![image](https://user-images.githubusercontent.com/6192491/125831115-69a86d97-b712-4e4d-94fb-a40c2367132c.png)

**Expected behavior**

We need a way to filter out short spans. I'm not sure, however, if the UI can be changed to show spans with a missing parent, or if the filtering method needs to be smarter and for example replace the parent id when the real parent is dropped (this currently seems hard to do, without support directly in the agent lib).

**Debug logs**

Two captured spans. The first is dropped by our filter. The second is stored but doesn't show up in the UI.

Click to expand

```
{
id: '492b3e17825250d4',
transaction_id: 'ae0b52e3a6c60f64',
parent_id: 'ae0b52e3a6c60f64',
trace_id: '413dfce475cad1107182651ea9abab46',
name: 'EXEC',
type: 'cache',
subtype: 'redis',
action: null,
timestamp: 1626368299979725,
duration: 1.616,
context: {
db: undefined,
http: undefined,
tags: undefined,
destination: { service: [Object], address: '127.0.0.1', port: 6379 },
message: undefined
},
stacktrace: undefined,
sync: false,
outcome: 'success',
sample_rate: 1
}

{
id: '6523f3b42ff3c563',
transaction_id: 'ae0b52e3a6c60f64',
parent_id: '492b3e17825250d4',
trace_id: '413dfce475cad1107182651ea9abab46',
name: 'GET registry.npmjs.org/jquery',
type: 'external',
subtype: 'http',
action: 'GET',
timestamp: 1626368299989391,
duration: 108.439,
context: {
db: undefined,
http: {
method: 'GET',
status_code: 200,
url: 'https://registry.npmjs.org/jquery'
},
tags: undefined,
destination: { service: [Object], address: 'registry.npmjs.org', port: 443 },
message: undefined
},
stacktrace: undefined,
sync: false,
outcome: 'success',
sample_rate: 1
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.