aws / aws/aws-xray-sdk-node

Very high CPU usage (at some point) with Postgres' capture

Open
#307 13 comments 0 reactions 1 assignee Claimed by @willarmiros View on GitHub
bug
Dominant language
JavaScript
Stars
280
Forks
157
PR merge metrics
No merged PRs in 30d

Description

Hi there,

We have been using X-Ray for a few weeks now and we have had a few issues with some of our Node processes using 100% of CPU usage for no apparent reason.

**Details of the issue**

CPU and memory usage increases abruptly.
Screenshot 2020-06-12 at 17 32 28
Screenshot 2020-06-12 at 17 33 05

- Even after stopping any request to go to this container, the CPU stays at 100%, up until a certain point. Once it took 23 minutes, in the example above it's been more than an hour and is still at 100% CPU.

There are *many* `Segment too large to send` warning logs at the moment where the load starts to spike.
Screenshot 2020-06-12 at 17 37 38

In the logs, the `trace_id` of the log itself (that we get with X-Ray's `getCurrentSegment()`) is different from the one in the log.
Screenshot 2020-06-12 at 17 40 24

Profiling the Node process, we can see that X-Ray functions are the ones taking all the CPU. On a ~11seconds profile, this is the result.
Screenshot 2020-06-12 at 17 16 58

**Questions**

- Do you have any idea of what might be the root cause?
- Do you have any idea of where to investigate next?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.