elastic / elastic/apm-agent-python

Investigate options and possible overhead of global string truncation

Open
#306 2 comments 0 reactions 0 assignees View on GitHub
agent-python
Dominant language
Python
Stars
431
Forks
239
Avg merge
5d 10h
Merged PRs (30d)
7

Description

To ensure some user errors don't lead to overblown payload sizes, we want to truncate all strings in the payload to a max length, similar to what we already to with strings that are stored as keywords in Elasticsearch. This could potentially be quite overhead-heavy, so we need benchmarks of different approaches

* explicitly wrap all strings in a `maybe_truncate` function that either returns the string itself or, if necessary, a truncated version
* loop over the payload and truncate all strings before serializing
* modify `JSONSerializer` to truncate strings

Contributor guide

Open the contributing guide

Research direction

Start with JSONSerializer and the payload serialization path described in the issue. Benchmark the three listed truncation approaches, including their overhead, and document the results and recommended approach; the issue does not specify the maximum length or a test target.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.