open-telemetry / open-telemetry/opentelemetry-python-contrib

Disk buffering span exporter to avoid data loss (existing Java feature)

Open
#3,299 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature-request
Dominant language
Python
Stars
1.1k
Forks
1.1k
Avg merge
4d 15h
Merged PRs (30d)
16

Description

What problem do you want to solve?

The current span exporter offerings may drop spans under high traffic if the buffer overflows.

Describe the solution you'd like

I am looking for a Python version of the disk buffering span exporters that were released in Java (https://github.com/open-telemetry/opentelemetry-java-contrib/tree/main/disk-buffering). These exporters store telemetry data in files which can be sent later on demand. There is a thread buffering the spans to a disk and a thread reading the spans from the disk.

Describe alternatives you've considered

Writing a custom span exporter that uses queueing/local storage to avoid overload, which would be complicated.

Additional Context

No response

Would you like to implement a fix?

None

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked Java disk-buffering implementation and the existing Python span exporter offerings. Define how spans are written to files, buffered by one thread, and read for later delivery by another. Done means a Python exporter avoids span loss when its in-memory buffer overflows and sends stored telemetry later.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
observability-sre
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.