open-telemetry / open-telemetry/opentelemetry-python
Unexpected behaviour observed after integrating with Otel
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 19
Description
Describe your environment Describe any aspect of your environment relevant to the problem, including your Python version, platform, version numbers of installed dependencies, information about your cloud hosting provider, etc. If you're reporting a problem with a specific version of a library in this repo, please check whether the problem has been fixed on main.
- Our service runs in docker.
- We use falcon server with gunicorn integration.
- We also use gevent patching for faster and non-blocking IO.
- We are using below versions of Otel:
opentelemetry-api 1.24.0 opentelemetry-distro 0.45b0 opentelemetry-exporter-otlp 1.24.0 opentelemetry-exporter-otlp-proto-common 1.24.0 opentelemetry-exporter-otlp-proto-grpc 1.24.0 opentelemetry-exporter-otlp-proto-http 1.24.0 opentelemetry-instrumentation 0.45b0 opentelemetry-instrumentation-asgi 0.45b0 opentelemetry-instrumentation-asyncio 0.45b0 opentelemetry-instrumentation-aws-lambda 0.45b0 opentelemetry-instrumentation-boto 0.45b0 opentelemetry-instrumentation-boto3sqs 0.45b0 opentelemetry-instrumentation-botocore 0.45b0 opentelemetry-instrumentation-dbapi 0.45b0 opentelemetry-instrumentation-falcon 0.45b0 opentelemetry-instrumentation-grpc 0.45b0 opentelemetry-instrumentation-logging 0.45b0 opentelemetry-instrumentation-pymemcache 0.45b0 opentelemetry-instrumentation-pymysql 0.45b0 opentelemetry-instrumentation-requests 0.45b0 opentelemetry-instrumentation-sqlite3 0.45b0 opentelemetry-instrumentation-system-metrics 0.45b0 opentelemetry-instrumentation-urllib 0.45b0 opentelemetry-instrumentation-urllib3 0.45b0 opentelemetry-instrumentation-wsgi 0.45b0 opentelemetry-propagator-aws-xray 1.0.1 opentelemetry-proto 1.24.0 opentelemetry-sdk 1.24.0 opentelemetry-semantic-conventions 0.45b0 opentelemetry-test-utils 0.45b0 opentelemetry-util-http 0.45b0
Steps to reproduce
Describe exactly how to reproduce the error. Include a code sample if applicable.
- Instrument falcon server in gunicorn master before server startup.
- Initialize Otel in post init worker.
- On a very higher load we are observing below issue:
- Our service made a db query Q1 and query Q2 at the same time. The results of Q1 and Q2 are getting interchanged. Suspect over here is that the db connection is somehow getting shared.
What is the expected behavior?
What did you expect to see? - No db connection sharing should happen.
What is the actual behavior?
What did you see instead? - We are suspecting db connection sharing.
Additional context
Add any other context about the problem here.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported high-load case using Falcon with Gunicorn, gevent patching, and the listed OpenTelemetry versions, following the issue's master and worker initialization steps. Investigate whether database connections are shared between workers or concurrent requests; done means simultaneous Q1 and Q2 consistently return their own results. No repository files or tests are named.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- backend, databases, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100