open-telemetry / open-telemetry/opentelemetry-python
Sporadic Connection Errors on Azure containers apps/jobs
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 19
Description
Describe your environment
OS: Various Linux distros (different base images used)
Python version: Python 3.11/3.12
SDK version: 1.32.1
What happened?
I'm using logfire as my logging library, which is a wrapper on top of the opentelemetry sdk.
We run microservices on azure container apps/jobs.
Sometimes, especially with jobs, the logging process fails with the following error:
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))
ERROR:opentelemetry.sdk.metrics._internal.export:Exception while exporting metrics
This is very sporadic, one job can log everything successfully, one won't be able to log at all, and some would stop mid-way through the run, leading to partial spans.
I've discussed this issue with the logfire team here and they claim it's not an issue with their backend.
I'm still not sure that's 100% accurate but I thought maybe here I'll find an idea as to why it happens.
Steps to Reproduce
If it helps, here's how I setup my logfire configuration (it sets up otel behind the scenes):
logfire.configure(
send_to_logfire="if-token-present",
token=settings.LOGFIRE_TOKEN,
service_name='SomeJob',
environment=settings.ENV_TYPE,
console=logfire.ConsoleOptions(min_log_level="trace", show_project_link=False),
advanced=logfire.AdvancedOptions(base_url="https://logfire-api.pydantic.dev")
)
Expected Result
logging is consistent and the connection doesn't interrupt midway.
Actual Result
sporadic connection errors.
Additional context
No response
Would you like to implement a fix?
None
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no repository files or tests; start by reproducing the sporadic export failure in Azure Container Apps or Jobs with the shown Python and Logfire configuration. Compare successful and failed runs, and review the linked Logfire discussion; done means identifying the cause of the connection resets and establishing consistent metric and span export.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, python
- Domain
- cloud, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100