open-telemetry / open-telemetry/opentelemetry-python-contrib
`requests` instrumentation causes out of memory errors on Google Cloud Run
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.1k
- Forks
- 1.1k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 16
Description
Environment
I'm working with a Python (3.8) gRPC server deployed to Cloud Run. I've installed the OpenTelemetry basics & added the gRPC and Redis instrumentation from this repo, but am having issues when adding the requests instrumentation. I've installed the following OpenTelemetry packages:
opentelemetry-api==1.5.0
opentelemetry-exporter-gcp-trace==1.0.0
opentelemetry-instrumentation==0.24b0
opentelemetry-instrumentation-grpc==0.24b0
opentelemetry-instrumentation-redis==0.24b0
opentelemetry-instrumentation-requests==0.24b0
opentelemetry-propagator-gcp==1.0.0
opentelemetry-sdk==1.5.0
opentelemetry-semantic-conventions==0.24b0
opentelemetry-util-http==0.24b0
Steps to reproduce
- Install
opentelemetry-instrumentation-requests - Call
RequestsInstrumentor().instrument()when instrumenting application - Deploy to Google Cloud Run
What is the expected behavior?
I would expect instrumenting the requests module to result in normal deployment & traced network requests.
What is the actual behavior?
The container running the server exceeds memory limits during deploy and fails to start. I've tried increasing the configured memory limit, but the issue persisted. Normal memory usage of this container is ~160-170MB, and limits of 256MB and 512MB have both resulted in failed deployment.
Additional context
This problem does not happen locally - checking docker stats shows expected memory usage. Additionally, I've been unable to find any reports of similar issues, however I don't know what would be incorrect in this configuration - I've basically just followed the example in the docs.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the listed OpenTelemetry package versions and the RequestsInstrumentor().instrument() call, then reproduce the deployment on Google Cloud Run while comparing memory with and without requests instrumentation. Use the requests instrumentation documentation and docker stats observations as the initial reference. Done means the cause of the Cloud Run memory failure is identified and a verified fix or limitation is established.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, grpc, python, redis
- Domain
- backend, cloud, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100