open-telemetry / open-telemetry/opentelemetry-python
Open telemetry auto instrumentation log flush error
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 19
Description
Describe your environment
OS: Windows 11
Python version: 3.11.9
SDK version: 1.35.0
API version: 1.35.0
What happened?
I have a very basic FastAPI that I am orchestrating with dotnet aspire (I don't think the aspire part is related, but mentioning it to be sure). I am using the zero code instrumentation to export all logs, traces, and metrics to the aspire dashboard, however, in log,s I keep getting this error non-stop:
Traceback (most recent call last):
File "D:\repos\...\PythonApp\.venv\Lib\site-packages\opentelemetry\sdk\_shared_internal\__init__.py", line 152, in _export
self._exporter.export(
File "D:\repos\...\PythonApp\.venv\Lib\site-packages\opentelemetry\sdk\_logs\_internal\export\__init__.py", line 98, in export
self.out.flush()
OSError: [Errno 22] Invalid argument
The error starts happening shortly after the app starts, and even before the app receives a single HTTP request.
I can see that the app is started with this command:
D:\repos...\PythonApp.venv\Scripts\opentelemetry-instrument.exe --traces_exporter otlp --logs_exporter console,otlp --metrics_exporter otlp
D:\repos...\PythonApp.venv\Scripts\python.exe main.py
I have some auto instrumentation libraries installed such as: "opentelemetry-instrumentation-fastapi". I can paste them all here if it will be useful. I am not using any manual instrumentation.
To be clear: everything is still working, both the console logger and the aspire dashboard. I can see all the traces, metrics, and logs in the dashboard.
And this error only appears on the Aspire dashboard, where the structured logs are supposed to appear, and not in the console, and that makes sense since the error seems to be coming from the exporter.
Steps to Reproduce
Start an auto-instrumented FastAPI that exports logs using grpc protocol to some compatible visualizer such as Grafana, Sentry, Azure App Insights or the Dotnet Aspire Dashboard (I only tested it with the Aspire dashboard, but I think it would be the same in other ones too)
Expected Result
I don't want to see the errors showing up every time it tries to flush. Everything is still working, but it's pretty annoying since it makes it impossible to see anything else on the dashboard.
Actual Result
Non-stop error message from the log exporter.
Additional context
No response
Would you like to implement a fix?
None
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in opentelemetry/sdk/_logs/_internal/export/init.py at the exporter's self.out.flush() call, then reproduce the issue with an auto-instrumented FastAPI app using the grpc logs exporter and console,otlp configuration. Done means the exporter continues sending logs to the dashboard without repeatedly emitting the OSError during flush.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- fastapi, grpc, python
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100