open-telemetry / open-telemetry/opentelemetry-cpp
[SDK] BatchLogRecordProcessor still drains the queue in a tight loop, inconsistent with BatchSpanProcessor fix
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.4k
- Forks
- 632
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 75
Description
Describe your environment
- Platform: macOS
- opentelemetry-cpp version: main branch
- Component:
sdk/src/logs/batch_log_record_processor.cc - BatchLogRecordProcessor configuration:
max_queue_size = 8192max_export_batch_size = 2048schedule_delay_millis = 5000
Steps to reproduce
- Configure
BatchLogRecordProcessorwith aLogRecordExporterthat counts export calls and records per call. - Produce log records at a constant rate (e.g. 5,000 logs/s) for 10 seconds.
- Observe the number of export requests and the average number of logs per request.
- Produce 50,000 log records without calling
ForceFlush().
What is the expected behavior?
With max_export_batch_size = 2048, exports should be close to 2048 records each. After #4466, BatchSpanProcessor behaves this way: on a normal wakeup it exports at most one batch, and only ForceFlush() / Shutdown() drain the entire buffer.
BatchLogRecordProcessor should follow the same semantics.
What is the actual behavior?
BatchLogRecordProcessor::Export() still drains the whole buffer in a tight loop:
void BatchLogRecordProcessor::Export()
{
do {
bool notify_force_flush =
synchronization_data_->is_force_flush_pending.exchange(false, std::memory_order_acq_rel);
if (notify_force_flush) {
num_records_to_export = buffer_.size();
} else {
num_records_to_export =
buffer_.size() >= max_export_batch_size_ ? max_export_batch_size_ : buffer_.size();
}
// ... consume & export ...
} while (true);
}
And the worker wakes up as soon as the buffer is non-empty:
synchronization_data_->cv.wait_for(lk, timeout, [this] {
...
return !buffer_.empty();
});
Measured result
| Metric | Value |
|---|---|
| Logs produced | 50,000 |
| Export calls | 2,332 |
| Average batch size | 21.4 |
| Full batches (>=2048) | 1 |
| Tiny batches (<100) | 2,331 |
The first export is 2048 records; every subsequent export in the same wakeup is ~20–25 records because the worker keeps draining the buffer instead of waiting for the next full batch.
Additional context
- #4449 reported the same problem for
BatchSpanProcessor. - #4466 fixed the trace path by changing the worker wait predicate and making
Export()drain the buffer only forForceFlush()/Shutdown(). - The logs path (
sdk/src/logs/batch_log_record_processor.cc) still uses the old implementation and should be aligned with the trace path.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with sdk/src/logs/batch_log_record_processor.cc and compare its behavior with the BatchSpanProcessor changes from #4466. Reproduce the export-call and batch-size measurements described in the issue, then verify that normal wakeups export one batch while ForceFlush() and Shutdown() drain the buffer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100