elastic / elastic/apm-agent-python
Investigate options and possible overhead of global string truncation
- Dominant language
- Python
- Stars
- 431
- Forks
- 239
- Avg merge
- 5d 10h
- Merged PRs (30d)
- 7
Description
To ensure some user errors don't lead to overblown payload sizes, we want to truncate all strings in the payload to a max length, similar to what we already to with strings that are stored as keywords in Elasticsearch. This could potentially be quite overhead-heavy, so we need benchmarks of different approaches
* explicitly wrap all strings in a `maybe_truncate` function that either returns the string itself or, if necessary, a truncated version
* loop over the payload and truncate all strings before serializing
* modify `JSONSerializer` to truncate strings
Contributor guide
Research direction
Start with JSONSerializer and the payload serialization path described in the issue. Benchmark the three listed truncation approaches, including their overhead, and document the results and recommended approach; the issue does not specify the maximum length or a test target.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100