open-telemetry / open-telemetry/opentelemetry-cpp
[BUG] Elasticsearch synchronous Export can block forever when an injected HTTP client never reports a terminal state
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.4k
- Forks
- 632
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 75
Description
What happens
The synchronous Elasticsearch export path waits with no deadline of its own
(exporters/elasticsearch/src/es_log_record_exporter.cc):
bool waitForResponse()
{
std::unique_lock<std::mutex> lk(mutex_);
cv_.wait(lk, [this] { return completion_ != CompletionState::Pending; });
return completion_ == CompletionState::Success;
}
Export() calls that and then session->FinishSession(). The predicate can only become
true from OnResponse or from one of the terminal OnEvent states, so the wait depends
entirely on the HTTP client reporting one of them.
Why it matters
The exporter takes an injected HttpClient, and that is a supported public surface, not a
test seam. Nothing in the EventHandler interface promises that a client which accepts a
handler will always deliver a terminal event. A client that accepts the request and then
returns without calling back, because a thread died, a socket was reused, or a wrapper
swallowed an error, leaves Export() blocked for the life of the process.
The caller has no way out. Export() takes no timeout, and the thread is inside the
exporter, so a Shutdown() from elsewhere cannot release it either.
The bundled curl client does report a terminal state on every path I traced, so this is
about injected clients rather than the default configuration.
Suggested shape
Give the handler its own absolute deadline rather than trusting the transport:
bool waitForResponse(std::chrono::steady_clock::time_point deadline)
{
std::unique_lock<std::mutex> lk(mutex_);
cv_.wait_until(lk, deadline, [this] { return completion_ != CompletionState::Pending; });
return completion_ == CompletionState::Success;
}
with the deadline derived from the configured response timeout. A wait that expires is a
failed export, which is the same answer the caller gets today from a terminal error event,
so no successful path changes.
Context
Pre-existing, and not something #4331 or #4337 introduces. #4331's comment on this wait
already states the dependency in as many words, which is what prompted writing it up
separately rather than widening either pull request.
Related: #4359 covers the same class of gap in Shutdown, where the timeout parameter is
accepted and ignored.
Happy to send a patch if the shape above looks right.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in exporters/elasticsearch/src/es_log_record_exporter.cc, reading Export() and waitForResponse() alongside the configured response-timeout handling. Make the wait use an absolute deadline and treat expiration as a failed export, then verify that terminal callbacks still produce the existing success or failure results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, elasticsearch
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100