elastic / elastic/apm-server

[RUM] traceId randomness is not guaranteed when page is crawled by googlebot

Open
#3,922 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
1.3k
Forks
543
Avg merge
1d 18h
Merged PRs (30d)
109

Description

+ Traces from pages crawled by bots (confirmed Goolgebot) has same trace id generated every time a trace is generated. This results in issues on the APM UI on the Service maps and also would be error prone when it comes to making sampling decisions if we cannot guarantee the uniqueness of the id.

There is not much we can do from the Agent side to guarantee the generation of random id from the agent side as
1. The random generator utilities like `Math.random` and `window.crypto.getRandomValues` are over-riden by the bots and will always be same.
2. Even relying on current time like `Date.now` and `Performance.now` also posses some issues as timestamps are clamped by 9-10 seconds during the last test on [Google search console](https://search.google.com/search-console/).

While having this issue discussion, we came to a general agreement on identify the transactions from bots and providing a way to discard these transactions if users find these less helpful. An idea was to add a field to the `transaction` schema to identify its coming from the bot.

```
transaction.isBot = true
// or
transaction.origin = "bot"
```

APM UI will exclude these transactions by default to fix the performance issues in Service Maps. /cc @graphaelli

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.