open-telemetry / open-telemetry/opentelemetry-java
BatchLogRecordProcessor's exportCurrentBatch wait for export finish, which limited logData to be exposed.
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 2.5k
- Forks
- 1k
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 58
Description
Is your feature request related to a problem? Please describe.
Our application has a high throughput of logData, using BatchLogRecordProcessor as a processor to batch output, we find that throughput is limited and CPU & memory usage are not high. After some investigation, we find that bottleneck is in BatchLogRecordProcessor.
BatchLogRecordProcessor has only one work and batch export pending on export result, so throughput is limited.
Describe the solution you'd like
Support multiple worker for BatchLogRecordProcessor, and worker number is configurable.
Describe alternatives you've considered
NA
Additional context
NA
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with BatchLogRecordProcessor.java at the export wait location linked in the issue. Trace how batches are queued and exported, then determine how configurable multiple workers should coordinate; done means the processor supports the requested worker count without the current single-export bottleneck.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100